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Executive Summa 


r 



The evidence is abundantly clear that a college degree is 
essential to economic success and social mobility in the 21st 
century, especially for low-income students and students of 
color, who historically have been left out of our higher educa- 
tion system. 1 However, many speculate about the value and 
outcomes of specific programs and institutions — in terms of 
both supporting students through to graduation and providing 
them with sufficient economic and noneconomic payoff. The 
information available today is inadequate and simply leaves 
the public wondering about answers to key questions about 
college access, progression, completion, cost, and outcomes. 

The Institute for Higher Education Policy has partnered with 
the Bill & Melinda Gates Foundation to develop a Metrics 
Framework (see Table ESI) built on a decade of research and 
experimentation by the field. 2 Recognizing the pressing need 
for better data, institutional and state initiatives have imple- 
mented a series of voluntary data collections to fill the gaps left 
by federal data systems in particular. We analyzed the metrics 
and definitions these voluntary initiatives use, along with data 
specifications in national and state data collections, to identify 
points of consensus in the field. The resulting key metrics fall 
into three major categories: 

• Performance metrics measuring institutional performance 
related to student access, progress, completion, cost, and 
post-college outcomes 

• Efficiency measures considering how resources impact 
college completion, driven by increased interest in college 
costs and affordability 

• Equity metrics seeking to include all students and accu- 
rately represent the higher education experience of popula- 
tions that are underserved and may be “invisible" in other 
data collections 


The field needs a core set 
of comprehensive and 
comparable metrics to 
answer critical questions 
about who attends college, 
who succeeds in and after 
college, and how college is 
financed. Importantly, to 
advance goals of social 
mobility and equity, the 
metrics must provide infor- 
mation specifically on how low-income and other underserved 
students fare. The metrics selected for the framework aim to 
measure each element as accurately and comprehensively as 
possible while balancing field convergence and data avail- 
ability and feasibility. 

We should no longer rely on the fortitude, creativity, and will- 
ingness of a select set of institutions and states to produce this 
information, but rather should incorporate these metrics into 
federal and state data systems. Doing so will make the data 
available for all students in all institutions, not only those that 
voluntarily collect and report it. These government data 
systems can make the results widely available to and usable 
by the public and policymakers, not to mention institutions 
themselves, thereby creating a transparent postsecondary 
system that facilitates effective policy and practice, and 
informed choices. 


We should no longer rely 
on the fortitude, creativity, 
and willingness of a 
select set of institutions 
and states to produce 
this information, but 
rather should incorporate 
these metrics into federal 
and state data systems. 
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Table ESI: Recommended Metrics and Definitions Along With Availability in Federal Data Sources (Integrated Postsecondary Education 
Data System [IPEDS] and National Student Loan Data System [NSLDS]) 



Key Performance Indicator 

Key Performance Indicator Definition 


Enrollment 

Twelve-month headcount that includes all undergraduate students who enroll at any point during the calendar year 


Credit Accumulation 

The percentage of students earning sufficient credits toward on-time completion in their first year 


Credit Completion Ratio 

The number of credits completed, divided by the number of credits attempted by first-year students 


Gateway Course Completion 

The percentage of students completing college-level, introductory math and English courses tracked separately in their first year 


Program of Study Selection 

The percentage of students in a cohort who demonstrate a program of study selection by taking nine credits (or three courses) in a meta-major in 
the first year 


Retention Rate 

The percentage of students in a cohort who are either enrolled at their initial institution or transfer to a longer program at the initial or subsequent 
institution, calculated annually up to 200% of program length 


Persistence Rate 

The percentage of students in a cohort remaining enrolled or earning a credential at their initial or subsequent institution, measured annually up 
to 200% of program length 


Transfer Rate 

The percentage of students in a cohort who transfer into longer programs at the initial or subsequent institution (s), up to 200% of program length 

LU 

Graduation Rate 

The percentage of students in a cohort who earn the credential sought at their initial institution, up to 200% of program length 

o 

z 

< 

Success Rate 

The percentage of students in a cohort who either graduate with the credential initially sought at the initial institution or transfer to a longer 
program at the initial or subsequent institution (s), up to 200% of program length 

cr 

O 

Li- 

CC 

Completers 

The number of students who complete a credential in a given year 

Net Price 

The average cost of attendance for an institution less all grant aid in a given year 

LU 

Q_ 

Unmet Need 

The average net price for an institution less the average expected family contribution (EFC) in a given year 


Cumulative Debt 

The median amount of debt student borrowers incur while attending an institution or program 


Loan Repayment Rate 

The percentage of borrowers in a cohort who make at least $1 of progress on their loan principal in a fiscal year, measured at one, three, five, 
and 10 years into repayment 


Cohort Default Rate 

The percentage of borrowers who enter repayment in a fiscal year and default within three fiscal years 


Graduate Education Rate 

The number and percentage of bachelor’s recipients enrolling in post-baccalaureate or graduate programs within one, five, and 10 (optional) 
years of completion 


Learning Outcomes 

Public display of student learning goals, assessments, and outcomes using the National Institute for Learning Outcomes Assessment’s (NILOA) 
Transparency Framework 


Employment Rate 

The percentage of former students with any reported earnings at one, five, and 10 years after exit from the institution 


Median Earnings 

The median annual earnings of former students one, five, and 10 years after exit from the institution (excludes zeros) 


Earnings Threshold 

The percentage of former students earning more than the median high school graduate salary ($25,000 in 2014; includes zeros) at one, five and 

1 0 years after exit from the institution 


Expenditures per Student 

Education and related expenditures per full-time equivalent (FTE) student based on 12-month enrollment 


Cost for Credits Not Completed 

The per-student expenditures for credits attempted but not completed by first-year students 


Cost for Completing Gateway 
Courses 

For all gateway course completers in a given year, the per-student expenditures associated with all developmental and gateway courses 
attempted before gateway completion, tracking English and math courses separately 

> 

Time to Credential 

The average time accumulated from first date of entry to the institution to date of completion for all completers in a given year 

o 

z 

Credits to Credential 

The average credits accumulated from the first date of entry to the institution to date of completion for all completers in a given year 

LU 

O 

LL 

Change in Revenue from 

Change in Retention 

The impact of changes in first-year retention rates from one cohort to another on tuition revenue available to the institution 

LL 

LU 

Cost of Excess Credits to 
Credential 

The per-student expenditures for excess credits to credential for all completers with excess credits in a given year 


Completions per Student 

The number of completions divided by the number of FTE students (based on 12-month enrollment) in a given year expressed as completions 
per 100 FTE 


Student Share of Cost 

The percentage of education and related expenditures covered by net student tuition revenue versus public subsidies in a fiscal year 


Expenditures per Completion 

Education and related expenditures divided by the number of completions in a fiscal year 


Enrollment Status 

First-time, transfer-in, or continuing students 


Attendance Intensity 

Full time and part time, determined by the institution based on the number of credit hours taken 


Credential-Seeking Status 

Certificate-, associate’s-, bachelor’s-, or noncredential-seeking students 


Program of Study 

Six-digit Classification of Instructional Program (CIP) codes and reported for seven meta-majors 

>- 

Academic Preparation 

Institutions classify students as “not college ready” and “college ready” in math and English as defined by institutional standards 

=) 

o 

LU 

Economic Status 

Pell Grant receipt as proxy for low-income or economic status 

Race/ethnicity 

Current IPEDS categories: Hispanic or Latino, American Indian or Alaska Native, Asian, Black or African-American, Native Hawaiian or Other 
Pacific Islander, White, Two or more races, Nonresident alien, and Race/ethnicity unknown 


Age 

Collected by date of birth, if available; otherwise reported by three categories: 19 and under, 20-24, 25 and over 


Gender 

Male, female, or other 


First-Generation Status 

Students whose parents’ highest education level is some college but no degree or below (e.g., some college, no degree; vocational or technical 
training; high school diploma or equivalent; did not complete high school) 


Note: These metrics measure undergraduate populations only. 

Key: Available with minor modifications needed Available with moderate modifications needed ■ I Available with major modifications needed WM Not available 
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The Metrics Framework is designed to be accessible to anyone interested in and passionate 
about improving current data collections to better student outcomes. This technical report aims to 
define each of the recommended metrics, demonstrate how the field converged on the metrics, 
and explore ways in which students, policymakers, and institutions can use the metrics to inform 
college decision making, design policies, and improve student success. Readers are encouraged 
to use this report to learn where in the field there is convergence on higher education data reporting 
and recommendations for how to calculate each of the metrics. 

The subsequent chapters cover cross-cutting criteria for students and institutions as well as each 
of the three major metric categories: performance, efficiency, and equity. In the performance and 
efficiency metric chapters, we organize the 31 metrics by the corresponding metric type: access, 
progression, completion, cost, and post-college outcomes. For each of the 31 metrics, we present 
a table that summarizes the metric definition, population of students included in the metric, recom- 
mended disaggregates (all 10 are defined in Chapter 5), and suggested submetrics. Institutions 
can use submetrics to drill into their data further for institutional improvement and student success. 
Finally, supporting each table is a description of how the field has measured this construct over 
the past several years and a discussion of how institutions, policymakers, and students can use 
the metric. 



Chapter 1: 

Introduction and Overview 

This chapter outlines the current state of higher education data, the purpose of the 
Metrics Framework, and the goal of the report. 


1.1 



Chapter 2: 

Cross-Cutting Criteria: Students and Institutions 2.1 

This chapter details cross-cutting definitions and cohort specifications that impact many 
of the performance metrics. Data parameters include the following: 

Twelve-Month Enrollment Population 2.2 

Enrollment Status 2.2 

Attendance Intensity 2.2 

Credential-Seeking Status 2.2 

Key Institutional Characteristics 2.4 
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Chapter 3: 

Performance Metrics 3.1 

This chapter includes all performance metrics, organized by metric type. 

The chapter describes each performance metric in detail. Metrics covered 
in this chapter include the following: 

Access 3.2 

Enrollment 3.2 

Progression 3.3 


Credit Accumulation 3.3 
Credit Completion Ratio 3.4 
Gateway Course Completion 3.5 
Program of Study Selection 3.6 
Retention Rate 3.7 
Persistence Rate 3.8 


Completion 3.10 

Outcome Rates (Transfer Rate, Graduation Rate, Success Rate) 3.10 
Completers 3.12 

Time and Credits to Credential 3.13 

Cost 3.14 

Net Price 3.14 
Unmet Need 3.16 
Cumulative Debt 3.17 

Post-College Outcomes 3.19 

Loan Repayment and Default Rates 3.19 
Graduate Education Rate 3.21 
Learning Outcomes 3.22 


Workforce Outcomes (Employment Rate, Median Earnings, Earnings Threshold) 3.23 
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Chapter 4: 

Efficiency Metrics 4.1 

This chapter discusses the efficiency metrics, organized by metric type. First, it reviews the 
overall use of efficiency metrics in the field as they relate to the increased focus on college cost 
and affordability. Second, it discusses the research and methodology of the Delta Cost Project, 
which underlies many of the efficiency metrics included in this framework. Finally, each efficiency 
metric is described in detail. Metrics covered in this chapter include the following: 

Access 4.4 

Expenditures per Student 4.4 


Progression 4.5 

Cost for Credits Not Completed 4.5 

Cost for Completing Gateway Courses 4.6 

Change in Revenue from Change in Retention 4.7 


Completion 4.8 

Cost of Excess Credits to Credential 4.8 
Completions per Student 4.9 

Cost 4.10 

Student Share of Cost 4.10 
Expenditures per Completion 4.1 1 



Chapter 5: 

Equity Metrics 5.1 

This chapter explains the equity metrics, which are key student characteristics that the framework 
recommends using to disaggregate the performance and efficiency metrics. These disaggregates 
are critical to promoting and enhancing equity in higher education. Field usage, convergence, 
research, and suggested use cases are explored in depth for the following characteristics: 

Academic Preparation 5.2 

Economic Status 5.3 


First-Generation Status 


5.5 


Program of Study 5.6 

Race/Ethnicity 5.6 

Gender 5.6 

Age 5.6 
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CHAPTER 1: 


Introduction and Overview 



The evidence is abundantly clear that a college degree is essen- 
tial to economic success and social mobility in the 21st century, 
especially for low-income students and students of color, who 
historically have been left out of our higher education system. 1 
However, many speculate about the value and outcomes of 
specific programs and institutions — in terms of both supporting 
students through to graduation and providing them with suffi- 
cient payoff for their investment. Better, more transparent data 
are needed to provide students, policymakers, and institutions 
with the information they need to answer important questions 
about college access, progression, completion, cost, and 
outcomes. Yet, with today’s outdated data systems, answers to 
these questions remain elusive: 

• How many low-income, first-generation, adult, transfer, and 
part-time students, who make up the new majority on today’s 
campuses, attend each college? 

• Do these students graduate? 

• How long does it take students, particularly students who 
enter with less academic preparation or fewer financial 
resources, to complete college? 

• Do the students who don’t graduate transfer, or do they drop 
out? 

• How much do students borrow, and can they repay these loans? 

• Can students find jobs in their chosen field, and how much 
do they earn? 

• What do students learn in college? 


Equitable access to and success in higher education relies on 
information that reflects the higher education experience of all 
students at all institutions, yet many of today’s students are 
missing or invisible in current data systems. Without answers 
to these key questions, progress toward equity and success 
for all students is quite simply stagnated — prospective students 
and policymakers will continue to be forced to make key deci- 
sions in a world lacking sufficient information. To advance the 
goals of social mobility and equity, the field needs a key set of 
comprehensive and comparable metrics that answer these 
critical questions about who attends college, who succeeds in 
and after college, and how college is financed. Specifically, the 
answers must provide information on how underserved 
students fare. Improved data that target student success will 
enable policymakers and institutions to help students — espe- 
cially students of color, low-income students, and first-genera- 
tion students — overcome barriers to college success, as well 
as empower the students themselves. 

Recognizing this problem, the Institute for Higher Education 
Policy has partnered with the Bill & Melinda Gates Foundation 
(BMGF) to develop a Metrics Framework (see Table 1-1) built 
on a decade of research and experimentation by the field. 
BMGF’s recent paper, Answering the Call, echoes the need for 
better and more complete data, calling for metrics that are 
reflective of all students, all institutions, and all outcomes. 2 It 
also outlines the proposed framework, which is designed to 
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Table 1-1: A Field-Driven Metrics Framework 



ACCESS 

PROGRESSION 

COMPLETION 

COST 

POST-COLLEGE OUTCOMES 

PERFORMANCE 

Enrollment 

Credit Accumulation 

Credit Completion Ratio 

Gateway Course Completion 

Program of Study Selection 

Retention Rate 

Persistence Rate 

Transfer Rate 

Graduation Rate 

Success Rate 

Completers 

Net Price 

Unmet Need 

Cumulative Debt 

Employment Rate 

Median Earnings 

Loan Repayment and Default 
Rates 

Graduate Education Rate 

Learning Outcomes 

EFFICIENCY 

Expenditures per Student 

Cost for Credits Not Completed 

Cost for Completing Gateway 
Courses 

Change in Revenue from 

Change in Retention 

Time/Credits to 

Credential 

Cost of Excess Credits to 
Credential 

Completions per Student 

Student Share of Cost 

Expenditures per 
Completion 

Earnings Threshold 

EQUITY 

Enrollment by (at least) 
Preparation, Economic 
Status, Age, Race/Ethnicity 

Progression Performance by 
(at least) Preparation, 

Economic Status, Age, Race/ 
Ethnicity 

Completion Performance 
by (at least) Preparation, 
Economic Status, Age, 
Race/Ethnicity 

Net Price and Unmet 
Need by (at least) 
Economic Status, 
Preparation, Age, Race/ 
Ethnicity 

Debt by (at least) 
Economic Status, Age, 
Race/Ethnicity, 
Completion Status 

Outcomes Performance and 
Efficiency by (at least) 
Preparation, Economic Status, 
Age, Race/Ethnicity, 

Completion Status 


Key Student Characteristics 


Key Institutional Characteristics 


Enrollment Status 
Attendance Intensity 
Credential-Seeking Status 
Program of Study 
Academic Preparation 


Economic Status 

Race/Ethnicity 

Age 

Gender 

First-Generation Status 


Sector 

Level 

Credential/Program Mix 
Size 

Resources 


Selectivity 

Diversity 

Minority-serving Institution (MSI) Status 
Post-traditional Populations 
Modality 


serve dual purposes. First, institutions can leverage these 
metrics to guide internal improvement efforts and better serve 
all students. Second, policymakers at the state and federal 
levels can incorporate these metrics into their respective data 
systems for wider consumer use and public consumption as 
well as to support the development of student-focused poli- 
cies. With these purposes in mind and informed by thorough 
research into the metrics that many states and institutions 
already are voluntarily using, this paper builds on Answering 
the Call to provide additional details on the metrics and defini- 
tions in the framework. 

To be clear, these metrics and definitions were not chosen in 
a vacuum. For more than 10 years, institutional and state 
initiatives have recognized the pressing need for better data 
and have implemented a series of voluntary data collections 
to fill the gaps left by federal data systems in particular. 3 We 
analyzed the metrics and definitions these voluntary initia- 
tives use, along with data specifications in national and state 
data collections, to identify points of consensus in the field. 
Appendix 1 shows which initiatives reviewed use each of the 
metrics included in the framework. The resulting metrics fall 
into three major categories: 

• Performance metrics measure institutional performance 
related to student access, progress, completion, cost, and 


post-college outcomes. Because many voluntary initiatives 
are designed to promote student access and success, most 
define and collect these types of performance indicators. 

• Efficiency metrics consider how resources impact college 
completion, driven by increased interest in attainment and 
affordability. Much of the methodology presented in this 
paper is derived from the Delta Cost Project, a leader in the 
development of comparable metrics on college costs. 

• Equity metrics seek to include all students, disaggregate by 
key student groups, and accurately represent the higher 
education experience of populations that are underserved 
and may be invisible in current data collections. Although 
some of these disaggregates are common practice in the 
field, this framework encourages increased disaggregation 
to support attainment goals for more underserved student 
groups. 

The metrics selected for the framework aim to measure each 
element as accurately and comprehensively as possible while 
balancing field convergence and data availability and feasi- 
bility. Institutions should be able to calculate the metrics as 
described using a variety of sources, including internal student 
information systems, the National Student Loan Data System 
(NSLDS), the College Scorecard, the National Student Clear- 
inghouse, and state earnings and employment records (if 
available). In the report, we note where institutions may have 
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difficulty obtaining data, but because data availability is contin- 
uously evolving, we also expect capacity and accuracy to 
continue to improve over time. Table 1-2 shows the metrics 
included in this framework and the associated definitions, with 
color coding to designate the current availability of these data 
in federal data sources. 

The field has spent the past decade refining a set of postsec- 
ondary metrics, with the goal of using these data to help 
advance student success. This experimentation has led to 
substantial consensus on what we should measure and how 



We should no longer rely on the fortitude, creativity, 
and willingness of a select set of institutions and 
states to produce this information, but rather should 
incorporate these metrics into federal and state data 
systems. 


Sidebox 1 : Data Privacy and Security 


Considering that student data are needed to calculate the 
metrics in this framework, protecting student privacy and 
ensuring the security of collected student data are essential. 
The Metrics Framework can be implemented as part of an 
aggregate or student-level collection, but the latter would be 
more flexible and easy to update over time. Institutions should 
take all appropriate measures under federal and state laws to 
ensure personally identifiable information is kept secure and 
confidential, while making aggregate results available trans- 
parently for consumers, policymakers, and the general public 
through state and federal collections. Secure access to post- 
secondary data is not an oxymoron, but an imperative to 
protect students — through both data privacy and transpar- 
ency about student outcomes. The U.S. Department of Educa- 
tion has a number of cybersecurity initiatives to ensure student 
data privacy and stop identity theft , 4 and makes available to 
institutions a number of resources 5 to build capacity and 
strengthen data protection . 6 Institutions should leverage these 
and other tools and resources to strengthen their systems and 
governance structures. 


in higher education. At this point, we should no longer rely on 
the fortitude, creativity, and willingness of a set of institutions 
and states to produce this information, but rather, should 
incorporate these metrics into federal and state data systems. 
Doing so will make the data available for all institutions, not 
only those that voluntarily collect and report it. These govern- 
ment data systems can make the results widely available to 
and usable by the public and policymakers, creating the trans- 
parent postsecondary system our students so desperately 
need, with respect for data privacy in security, as outlined in 
Sidebox 1. With this transparent system, policymakers can 
design effective policies and students can make informed 
choices. 
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Table 1-2: Recommended Metrics and Definitions Along With Availability in Federal Data Sources (Integrated Postsecondary Education 
Data System [IPEDS] and National Student Loan Data System [NSLDS]) 



Key Performance Indicator 

Key Performance Indicator Definition 


Enrollment 

Twelve-month headcount that includes all undergraduate students who enroll at any point during the calendar year 


Credit Accumulation 

The percentage of students earning sufficient credits toward on-time completion in their first year 


Credit Completion Ratio 

The number of credits completed, divided by the number of credits attempted by first-year students 


Gateway Course Completion 

The percentage of students completing college-level, introductory math and English courses tracked separately in their first year 


Program of Study Selection 

The percentage of students in a cohort who demonstrate a program of study selection by taking nine credits (or three courses) in a meta-major in 
the first year 


Retention Rate 

The percentage of students in a cohort who are either enrolled at their initial institution or transfer to a longer program at the initial or subsequent 
institution, calculated annually up to 200% of program length 


Persistence Rate 

The percentage of students in a cohort remaining enrolled or earning a credential at their initial or subsequent institution, measured annually up 
to 200% of program length 


Transfer Rate 

The percentage of students in a cohort who transfer into longer programs at the initial or subsequent institution (s), up to 200% of program length 

LLI 

Graduation Rate 

The percentage of students in a cohort who earn the credential sought at their initial institution, up to 200% of program length 

o 

z 

< 

Success Rate 

The percentage of students in a cohort who either graduate with the credential initially sought at the initial institution or transfer to a longer 
program at the initial or subsequent institution (s), up to 200% of program length 

cc 

o 

LL 

CC 

Completers 

The number of students who complete a credential in a given year 

Net Price 

The average cost of attendance for an institution less all grant aid in a given year 

LLI 

CL 

Unmet Need 

The average net price for an institution less the average expected family contribution (EFC) in a given year 


Cumulative Debt 

The median amount of debt student borrowers incur while attending an institution or program 


Loan Repayment Rate 

The percentage of borrowers in a cohort who make at least $1 of progress on their loan principal in a fiscal year, measured at one, three, five, 
and 10 years into repayment 


Cohort Default Rate 

The percentage of borrowers who enter repayment in a fiscal year and default within three fiscal years 


Graduate Education Rate 

The number and percentage of bachelor’s recipients enrolling in post-baccalaureate or graduate programs within one, five, and 10 (optional) 
years of completion 


Learning Outcomes 

Public display of student learning goals, assessments, and outcomes using the National Institute for Learning Outcomes Assessment’s (NILOA) 
Transparency Framework 


Employment Rate 

The percentage of former students with any reported earnings at one, five, and 10 years after exit from the institution 


Median Earnings 

The median annual earnings of former students one, five, and 10 years after exit from the institution (excludes zeros) 


Earnings Threshold 

The percentage of former students earning more than the median high school graduate salary ($25,000 in 2014; includes zeros) at one, five and 

1 0 years after exit from the institution 


Expenditures per Student 

Education and related expenditures per full-time equivalent (FTE) student based on 12-month enrollment 


Cost for Credits Not Completed 

The per-student expenditures for credits attempted but not completed by first-year students 


Cost for Completing Gateway 
Courses 

For all gateway course completers in a given year, the per-student expenditures associated with all developmental and gateway courses 
attempted before gateway completion, tracking English and math courses separately 

> 

Time to Credential 

The average time accumulated from first date of entry to the institution to date of completion for all completers in a given year 

o 

z 

Credits to Credential 

The average credits accumulated from the first date of entry to the institution to date of completion for all completers in a given year 

LU 

o 

LL 

Change in Revenue from 

Change in Retention 

The impact of changes in first-year retention rates from one cohort to another on tuition revenue available to the institution 

LL 

LLI 

Cost of Excess Credits to 
Credential 

The per-student expenditures for excess credits to credential for all completers with excess credits in a given year 


Completions per Student 

The number of completions divided by the number of FTE students (based on 12-month enrollment) in a given year expressed as completions 
per 100 FTE 


Student Share of Cost 

The percentage of education and related expenditures covered by net student tuition revenue versus public subsidies in a fiscal year 


Expenditures per Completion 

Education and related expenditures divided by the number of completions in a fiscal year 


Enrollment Status 

First-time, transfer-in, or continuing students 


Attendance Intensity 

Full time and part time, determined by the institution based on the number of credit hours taken 


Credential-Seeking Status 

Certificate-, associate’s-, bachelor’s-, or noncredential-seeking students 


Program of Study 

Six-digit Classification of Instructional Program (CIP) codes and reported for seven meta-majors 

>- 

Academic Preparation 

Institutions classify students as “not college ready” and “college ready” in math and English as defined by institutional standards 

=) 

o 

LU 

Economic Status 

Pell Grant receipt as proxy for low-income or economic status 

Race/ethnicity 

Current IPEDS categories: Hispanic or Latino, American Indian or Alaska Native, Asian, Black or African-American, Native Hawaiian or Other 
Pacific Islander, White, Two or more races, Nonresident alien, and Race/ethnicity unknown 


Age 

Collected by date of birth, if available; otherwise reported by three categories: 19 and under, 20-24, 25 and over 


Gender 

Male, female, or other 


First-Generation Status 

Students whose parents’ highest education level is some college but no degree or below (e.g., some college, no degree; vocational or technical 
training; high school diploma or equivalent; did not complete high school) 


Note: These metrics measure undergraduate populations only. 

Key: Available with minor modifications needed Available with moderate modifications needed WM Available with major modifications needed Not available 
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CHAPTER 2: 


Cross-Cutting Criteria: 
Students and Institutions 



This chapter details the cross-cutting definitions and cohort specifications that 
impact many of the performance metrics included in the framework. Parameters 
include the following: 


Twelve-month enrollment population 2.2 

Enrollment status 2.2 

Attendance intensity 2.2 

Credential-seeking status 2.2 

Key institutional characteristics 2.4 
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Cohort Specifications 

Many metrics in this framework measure performance for a 
specific group, or cohort, of students. In an attempt to build 
consistency across metrics, the framework defines cohorts as 
similarly as possible for each metric, and this chapter discusses 
the core cohort recommendations that apply to multiple 
metrics. For each decision point, we summarize the recom- 
mendation, evidence from the field that led us to that recom- 
mendation, and possible ways that the data can be used. 

These definitional decisions are centered on the principle of 
counting all students. In many cases, this framework expands 
on the students included in the Integrated Postsecondary 
Education Data System (IPEDS) graduation-rate cohort to be 
more inclusive of all students and reflect the demographics, 
attendance, and mobility patterns of 21st-century students. 
Grounded in the work and innovation of numerous voluntary 
initiatives, the framework offers guidance on how to define the 
cohort population and create separate cohorts based on 
enrollment status, attendance intensity, and credential-level 
sought. 

Cohort Population: 12-Month Enrollment 

The framework recommends defining the population for most 
progression and completion metrics as all students who enter 
an institution during a 12-month period (12-month cohort, also 
known as a full-year cohort), instead of only students who 
enter the institution in the fall (fall cohort). This specification 
enables the metrics to capture the one in four students who 
start college outside the fall term, a particular issue in the 
community college and for-profit sectors, where about 35 
percent and 45 percent of students begin at times other than 
the fall, respectively. 1 

Field Usage and Convergence 

While many voluntary initiatives — including the Voluntary 
Framework of Accountability (VFA) and Achieving the Dream 
(ATD) — use fall-enrollment cohorts, moving toward 12-month 
cohorts aligns with the goal of counting all students. Further, 
IPEDS and Complete College America (CCA) have set a prec- 
edent for 12-month cohorts by allowing some institutions to 
use full-year instead of fall cohorts, depending on their 
academic calendar system. 2 Also, 32 members of the Postsec- 
ondary Data Collaborative have supported this switch to 
12-month cohort reporting in order to capture more nontradi- 
tional students. 3 

Use Cases 

Expanding cohorts to capture students beginning at any point 
during the year will provide institutions with a more compre- 
hensive picture of student progression and completion, 
capturing outcomes for the 6 million students who enter insti- 
tutions each year at times other than the traditional fall 
semester. While the late-year entrants will have slightly less 


time to count as completers during a specified period (e.g., 
150 percent of time), the benefit of adding these students into 
the calculation outweighs this downside. Because these 
students already are included in the 12-month enrollment 
counts for IPEDS, incorporating them into progression and 
success cohorts should not unduly increase burden. 

Enrollment Status and Attendance Intensity 

The framework recommends separating each cohort by enroll- 
ment status (first-time or transfer-in) and attendance intensity 
(full-time or part-time). This approach creates the following 
four distinctive cohorts: first-time full-time (FTFT), first-time 
part-time (FTPT), transfer full-time (TFT), and transfer part-time 
(TPT), as determined by students’ status at entry. All four of 
these cohorts should be defined in each credential-seeking 
category (see next section for more details on defining creden- 
tial-seeking status). 

Field Usage and Convergence 

Current IPEDS graduation rates track only FTFT students, 
thus excluding about 45 percent of today’s degree- and 
certificate-seeking college entrants. 4 The rates often are criti- 
cized for their narrow scope, so 13 voluntary initiatives — 
including CCA and VFA — include transfer or part-time 
students in their completion rates in a variety of ways to be 
more inclusive of today’s students. 5 The 2015 release of 
College Scorecard data included both the IPEDS graduation 
rate and rates for all Title IV recipient students, using National 
Student Loan Data System (NSLDS) data. While these data 
are imperfect, this release shows a movement toward non- 
first-time, full-time cohort usage. Also, the new IPEDS 
Outcome Measures component used the same four cohorts 
this framework recommends. 6 

Use Cases 

By including transfer and part-time students in the cohorts, 
these data present a more complete view of progression and 
completion for all students enrolled at the institution, compared 
with the IPEDS FTFT graduation rates, which have long and 
widely been criticized by the field. The inclusion of these 
multiple cohorts reinforces the commitment to supporting all 
students enrolled in higher education through to completion. 

Credential Level Sought 

The framework recommends distinguishing cohorts by 
credential level sought — including non-credential-seeking, 
certificate-seeking, associate’s-seeking, and bachelor’s- 
seeking students — because these degree types differ in 
expected time to completion. Data show that students who 
complete a degree or certificate tend to complete the creden- 
tial they initially sought, indicating they can accurately report 
their degree plans at entry. Among completers at their first 
institution, nearly 100 percent of certificate-seeking students, 
82 percent of associate’s-seeking students, and 98 percent of 
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bachelor’s-seeking students received their intended degree, 
as opposed to a credential of a different level (see Table 2-1 for 
more details). Also, data show that persistence and comple- 
tion rates vary substantially among students pursuing different 
credential types, highlighting the importance of measuring 
these student groups separately. 7 While some have noted 
challenges with identifying students’ credential level at entry, 
institutions now are required to report this information to the 
NSLDS to allow for tracking of student loan eligibility, which is 
limited to 1 50 percent of program length. This new compliance 
requirement should enhance the quality of data on students’ 
credential level to near complete coverage, as reported by the 
National Student Clearinghouse. 

Field Usage and Convergence 

As a required reporting element in NSLDS and in codebooks 
for voluntary initiatives such as CCA, Completion by Design, 
and the Student Achievement Measure, the field has demon- 
strated that institutions are capable of reporting data sepa- 
rately by the level of credential sought, including for certificates 
for some initiatives. Some have debated whether institutions 
can accurately determine student intentions at college entry in 
order to place them in a cohort. To examine this issue, we 
evaluated the trends in student degree plans, enrollment 
activity, and credential completion in a nationally representa- 
tive survey, the Beginning Postsecondary Students (BPS) 
study. In sum, these results — discussed in the bullets below — 
show that activity by students in their first year tends to match 
their intent. 

1 . Students who earn credentials at their first institution tend 
to earn the credential they initially sought: Some have 
posited that placing students in cohorts based on the level of 
credential initially sought — and measuring success as comple- 
tion of that credential — could undercount actual success 
rates, because students may complete other credentials, such 


as stackable credentials, even if they do not ultimately 
complete their longer, intended degree. However, analysis of 
BPS data shows that very few students who earn a credential 
at their first institution earn it at a level other than their initial 
program of entry. Students are most likely to complete the 
credential they sought or leave college altogether, but they are 
unlikely to complete a credential of a different level than the 
one they initially sought. Further, most students, especially in 
certificate and bachelor’s programs, earn the degree they 
initially sought at their first institution rather than at a subse- 
quent institution. For example, only 1 percent of students in 
bachelor’s programs earned either a certificate or associate’s 
degree at their first institution, and only 4 percent of them 
earned a credential other than a bachelor’s degree anywhere. 
Additionally, only 4 percent of students in associate’s programs 
earned a certificate or bachelor’s degree in six years at their 
initial institution (see Table 2-1). 8 

2. Most students do not seek credentials only to receive 
financial aid: There remains some concern in the field that 
students who are enrolled in degree programs are desig- 
nated as such only in order to receive financial aid, not 
because they intend to complete a credential. Here, we 
examine several trends that seem to at least limit the scope 
of that problem. Using BPS, we found: (1) 65 percent of 
degree-seeking federal aid recipients who eventually drop 
out stay enrolled beyond their first year, suggesting that 
most of them intended to receive the degree sought, a trend 
consistent with students who do not receive aid; 9 and (2) 
degree-seeking students who do not complete are as likely 
to have borrowed as students who do complete and more 
than half (52 percent) of noncompleters borrow. While it may 
be feasible that students would declare false degree inten- 
tions in order to receive grant aid, it seems less likely that 
they would do so to take out loans, especially given that 
most noncompleters do persist beyond the first year. 


Table 2-1: Cumulative Attainment in Six Years at First Institution and Anywhere 


Degree program 

Attained 

bachelor’s degree 

Attained 

associate’s degree 

Attained 

certificate 

No degree, 
still enrolled 

No degree, 
transferred 

No degree, 
left without return 

Attainment at First Institution 

Certificate 

* 

* 

51% 

3% 

13% 

33% 

Associate’s degree 

1% 

9% 

3% 

8% 

31% 

38% 

Bachelor’s degree 

56% 

1% 

0% 

5% 

24% 

14% 

Not in a degree program 

4% 

7% 

6% 

9% 

36% 

38% 

Attainment Anywhere 

Certificate 

* 

2% 

52% 

9% 

N/A 

36% 

Associate’s degree 

11% 

18% 

6% 

18% 

N/A 

46% 

Bachelor’s degree 

63% 

3% 

1% 

12% 

N/A 

21% 

Not in a degree program 

16% 

7% 

8% 

21% 

N/A 

49% 


Source: IHEP analysis of Beginning Postsecondary Student Survey: 2004-09 data. 

Note: Rows may not add to 100 percent due to rounding. Asterisks (*) denote where estimates are unstable because the standard error represents more than 30 or 50 percent of the estimate. 
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3. Behaviorally defined cohorts likely omit many students 
who are legitimately seeking credits: Behaviorally defined 
cohorts base credential-seeking status on students’ course- 
taking behavior, rather than their stated degree intention, 
due to concerns about the accuracy of student self-reports. 10 
However, using BPS, we found that a considerable number 
of students who fail to meet early behavioral milestones, 
such as accumulating six to 12 credits in their first one to 
three years, do display persistence behavior, such as 
course-taking and completion, beyond the initial enrollment 
time frame (i.e., one to three years). In brief, students who 
fail to meet early behavioral milestones at community 
colleges do go on to complete credentials both at their initial 
institution (about 14 percent) and at subsequent institutions 
(about 8 percent) within six years. Additionally, about 10 
percent of these students continue enrollment at their initial 
institution, and about 31 percent transfer to a subsequent 
institution within six years. 11 It should be noted that including 
these students in the cohort does lower completion rates 
because students who do not gain early academic 
momentum are less likely to complete than those who attain; 
however, eliminating them from the cohort reduces the 
chances that institutions can or will intervene to help them 
on their pathway to success. Because of the goal to count all 
students and all outcomes, this iteration of the framework 
uses initial credential-level to maximize the scope of students 
included in cohorts. Further, as behaviorally defined cohorts 
can only be developed retrospectively, they are not condu- 
cive to improvement efforts, which need to track student 
cohorts in real-time in order to influence their outcomes. 12 
Finally, as discussed in the persistence and outcome rates 
chapters subchapters, all outcomes will be captured regard- 
less of the credential sought by the student. For example, an 
associate’s-seeking student who earns a certificate will 
count as persisting in the persistence rate, even though she 
will not count in the numerator of the outcome rate. 

Use Cases 

Because behaviors and outcomes are distinct, depending on 

students’ initial credential level, the framework recommends 

tracking each cohort separately based on the type of creden- 
tial sought. This disaggregation will allow institutions to under- 


stand how specific groups of students progress toward 
credentials and to identify challenges that impact success 
differently for students in different credential programs. Further, 
in the framework, students who transfer between credential 
levels at the initial institution or a subsequent institution would 
still be counted, so their progress would still be reported. 

Key Institutional Characteristics 

In addition to the commitment to count all students, the frame- 
work also seeks to include all institutions. For this reason, it 
recommends collecting key institutional characteristics that 
can help contextualize the student data and the mission of the 
institutions. Table 2-2 shows the institutional characteristics 
included in the framework. 


Table 2-2: Key Institutional Characteristics 


• Sector 

• Selectivity 

• Level 

• Diversity 

•Credential/Program mix 

• Minority-Serving Institution status 

•Size 

• Post-traditional populations 

• Resources 

• Modality 


Each characteristic has implications for understanding the 
context of the institution. Sector, level, and degree or program 
mix are commonly used in research to distinguish institutions 
at their most basic level: who controls the institution and what 
degree types are available. The size and resources of the insti- 
tution establish the fiscal framework of the institution — small, 
resource-rich schools operate in a very different environment 
than do larger, underresourced schools. The selectivity of the 
school reveals important information about the academic 
preparation of incoming students, while diversity and Minority- 
Serving Institution status demonstrate the demographics and 
history of a campus. Finally, modality is becoming increasingly 
important, as institutions expand their online learning capaci- 
ties. By understanding the structure of the institutions, student 
access, progression, completion, cost, and post-college 
outcomes can be contextualized as appropriate. These data 
can be obtained from IPEDS and will not cause additional 
reporting burden for the institutions. 
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CHAPTER 3: 


Performance Metrics 



This chapter describes the following set of performance metrics: 


Access 3.2 

Enrollment 3.2 

Progression 3.3 


Credit Accumulation 3.3 
Credit Completion Ratio 3.4 
Gateway Course Completion 3.5 
Program of Study Selection 3.6 
Retention Rate 3.7 
Persistence Rate 3.8 


Completion 3.10 

Outcome Rates (Transfer Rate, Graduation Rate, Success Rate) 3.10 
Completers 3.12 

Time and Credits to Credential 3.13 

Cost 3.14 

Net Price 3.14 
Unmet Need 3.16 
Cumulative Debt 3.17 

Post-College Outcomes 3.19 

Loan Repayment and Default Rates 3.19 
Graduate Education Rate 3.21 
Learning Outcomes 3.22 


Workforce Outcomes (Employment Rate, Median Earnings, Earnings Threshold) 3.23 
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ACCESS 


Enrollment 

Definition 

Twelve-month headcount that includes all undergraduate 
students who enroll at any point during the calendar year 

Population 

Twelve-month unduplicated undergraduate headcount by 
credential level and student enrollment status and attendance 
intensity 

Disaggregates 

Academic preparation, economic status (at entry), race/ 
ethnicity, gender, age, first-generation status, and program of 
study 


Submetrics 
for further 
analysis 


• Recruitment of underrepresented populations 

•Application start and completion among underrepresented 
populations 

• Financial aid application completion by underrepresented 
populations 

• Financial aid gap for underrepresented populations 
•Acceptance rates for underrepresented populations 
•Yield for underrepresented populations 

• Enrollment in Science, Technology, Engineering, and Math 
(STEM) fields by underrepresented populations 

• Date of application or enrollment relative to term start date 

• Dual or summer enrollment before first term 

• Co-enrollment in another institution ^ 


Field Usage and Convergence 
All of the reviewed initiatives consistently collect an enrollment 
metric, though most collect only fall enrollment because much 
of the Integrated Postsecondary Education Data System’s 
(IPEDS) reporting is based on fall counts. Not only are IPEDS’ 
fall enrollments disaggregated more thoroughly than IPEDS’ 
12-month enrollments, but retention and graduation rates for 
most institutions also are based on fall cohorts. While fall 
enrollments include only students who follow the traditional 
academic schedule, 12-month enrollments include every 
student, regardless of whether they first enroll in the fall, 
spring, or summer. This broader enrollment definition, used by 
Complete College America (CCA) and Predictive Analytics 
Reporting (PAR) Framework, increases total enrollment counts 
by about 25 percent and includes more nontraditional students 
whose enrollment and attendance patterns fall outside of past 
norms. 


and disaggregating the data by academic preparation, 
economic status, race/ethnicity, gender, age, first-generation 
status, and program of study. IPEDS currently includes some 
of these details (e.g., enrollment status, attendance intensity) 
in the fall enrollment survey but includes only race/ethnicity 
and gender in the 12-month survey. Fully disaggregating full- 
year enrollments will provide a more comprehensive view of 
access trends, especially for key demographics, including 
underprepared, low-income, and underrepresented minority 
populations. 

Use Cases 

By expanding the metric to include entrants throughout the 
year, institutions would include almost one in four more 
students, largely from the community college and for-profit 
sectors. 2 Enrollment is a foundational metric for this frame- 
work because 12-month enrollment also defines the cohort 
population used in many of the other metrics, such as outcome 
rates. It also provides a baseline of information about college 

access, allowing institutions to 
measure their effectiveness at 
enrolling diverse student popula- 
tions, to evaluate access over time, 
and to assess their campus diversity against their service 
area’s demographics. Policymakers use enrollment data to 
determine how effective institutions are at enrolling diverse 
student populations and can design policies to advance that 
agenda. Students and families use these data — such as 
whether there are students like them enrolled there — to help 
determine fit at an institution. 


initiatives measure Enrollment 


As such, the framework recommends the use of 12-month 
counts — both to measure enrollment trends and to define 
cohorts for other metrics, such as outcome rates, a change 
supported by many higher education policy organizations (for 
more detail, see the Postsecondary Data Collaborative 
comments on IPEDS Outcome Measures (OM) and their 
response to Sen. Lamar Alexander [R-TN]). 1 This framework 
recommends creating 12-month enrollment cohorts based on 
enrollment status, attendance intensity, and credential level, 
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PROGRESSION 


Credit Accumulation 

Definition 

The percentage of students earning sufficient credits toward 
on-time completion in their first year: 30 credits for full-time and 
15-credits for part-time students. Prior credits from Advanced 
Placement (AP), International Baccalaureate (IB), dual 
enrollment, and transfer are not counted, nor are noncredit 
remedial courses. Credit is earned based on institutional 
standards 

Population 

Twelve-month cohorts by credential level and student 
enrollment status and attendance intensity (e.g., first-time 
full-time [FTFT], transfer full-time [TFT], first-time part-time 
[FTPT], transfer part-time [TPT]) 

Disaggregates 

Academic preparation, economic status (at entry), race/ 
ethnicity, gender, age, first-generation status, and program of 
study (at entry) 


Submetrics 
for further 
analysis 


• Remedial course enrollment and completion (if applicable) 

• Average credit load per term or year 

• Summer or intersession credits earned 

• Enrolled at least half time (for part-time students) 

•Continuous enrollment ^ 


students are progressing to more fully inform institutional 
improvement efforts. (For an in-depth explanation of consider- 
ations around cohort determination, please see Chapter 2). 

Use Cases 

This metric is designed to help institutions and policymakers 
measure the extent to which students are progressing toward 
completion, and the disaggregates clarify which students are 
(and are not) gaining academic momentum early in order to 
determine what can be done to help more of these students 
succeed. Disaggregation is especially important for this metric 
because it better articulates the degree pathways for full-time 
or part-time and first-time or transfer students. While any given 
student may have specific reasons for taking more or less 
credits (e.g., program of study, personal finances), the average 
number of credits accumulated by entering students in the first 

year serves as an impor- 
tant institution-wide indi- 
cator of student progress. 


initiatives measure Credit Accumulation 


Field Usage and Convergence 

Drawing on research that demonstrates early credit accumula- 
tion as a key leading indicator of degree completion, 3 Achieving 
the Dream (ATD), CCA, the Voluntary Framework of Account- 
ability (VFA) , and six other initiatives 4 established the wide- 
spread use of this metric with hundreds of colleges in nearly 
all 50 states across the country. Following the most expansive 
collection of this metric in the field, the recommended 30- and 
15-credit thresholds align with CCA’s reporting requirements, 
which were recently revised to further encourage on-time 
completion for full-time students while recognizing that part- 
time students also need a reasonable, yet timely, pathway to 
success. 5 Although some initiatives suggest including all 
credits in this calculation, the framework excludes remedial 
courses here, per CCA and other organizations, because 
those credits do not count toward a credential. Also, a coali- 
tion of six organizations recently encouraged the adoption of 
corequisite remediation and other new models that support 
students to actively accrue credits toward their credentials — 
despite needing developmental education — adding further 
reason to not count separate developmental credits toward 
this progression metric. 6 

Building on the field’s work, the framework also recommends 
the addition of key disaggregates. As such, this framework’s 
proposed metric builds on CCA’s specifications by expanding 
to a 1 2-month cohort that incorporates nonfall entrants, adding 
cohorts for transfer students, and separating cohorts based 
on the level of credential sought. Reporting lag times are 
expected because spring entrants should receive a full year to 
accumulate credits. These changes align this progression 
metric with the enrollment and completion metrics in the 
framework and provide more detail about how specific 


It is also important to disaggregate by credential level, as 
students in various programs (certificate-seeking, associate’s- 
seeking, and bachelor’s-seeking) may tend toward different 
problems and solutions relative to academic progress. The 
field shows that academic preparation is among the most 
important disaggregates for this metric, because remedial 
requirements can slow student progress toward credit mile- 
stones and eventual completion and largely do not count 
toward a credential. Economic status is also an essential 
disaggregate because students must be able to afford and 
enroll in courses to earn credit. Further, active intervention by 
institutions can positively affect students’ level of preparation 
and financial situations. 

In order to support an institution’s ability to understand student 
momentum and progression, the framework highlights addi- 
tional submetrics, like average credit load and credit comple- 
tion ratio, as a way for colleges and universities to drill down 
into these metrics and develop strategies to address stalled 
students. The remedial enrollment and completion submetrics 
could be useful for institutions, as these submetrics highlight 
which students are affected and need additional support. 
Credit accumulation indicators have been incorporated into 
early warning systems and advising technology, like Civitas 
and Starfish, to make the data useful for students and advi- 
sors. Policymakers also have incorporated credit-based 
momentum measures into many outcome-based funding 
models to shape state funding. 7 
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PROGRESSION, continued 


Credit Completion Ratio 

Definition 

The number of credits completed, divided by the number of 
credits attempted by first-year students. Prior credits from AR 

IB, dual enrollment, and transfer are not counted. Credit is 
earned based on institutional standards. 

Population 

Twelve-month incoming cohorts by credential level and student 
enrollment status and attendance intensity (e.g., FTFT, FTPT, 

TFT, TPT) 

Disaggregates 

Academic preparation, economic status (at entry), race/ 
ethnicity, gender, age, first-generation status, and program of 
study (at entry) 

Submetrics 
for further 
analysis 

• Percentages of remedial courses among uncompleted credits 

•Percentage of D's, F’s, W', I’s among uncompleted credits 

• Percentage of uncompleted credits that were retakes 

•Percentage of D's, F’s, W’s, I’s in high-enrollment courses 

• Grade point average by term and year 

• Course engagement/interaction by course completion 

• Course format/modality by course completion 


Field Usage and Convergence 
CCA, the Consortium for Student Retention Data Exchange 
(CSRDE), the PAR Framework, and the VFA all use credit 
completion ratios to measure student momentum toward a 
credential. This framework follows a combination of VFA’s, 
CSRDE’s, and CCA’s definitions. For example, the metric 
includes remedial courses in the calculation even if they do not 
count toward degree requirements, which is consistent with 
VFA and CSRDE standards. Including, rather than omitting, 
these remedial courses provides a more complete analysis of 
academic momentum remedial courses; however, this frame- 
work’s proposed metric follows CCA guidelines for credit 
completion by counting D’s only if the institution accepts the 
grade as passing, while VFA and PAR Framework, on the other 
hand, exclude D’s in all cases. CCA’s guideline for treatment of 
D’s is more customized to individual institution’s circumstances. 


Use Cases 

The credit completion ratio improves institutional under- 
standing of credit accumulation and student academic 
momentum in the first year by focusing in on courses passed 
versus courses attempted. Research shows that higher credit 
accumulation ratios in the first year are correlated with ultimate 
credential completion, so the measure can be a useful tool to 
discover students’ academic setbacks and allow for early 
interventions . 8 The academic preparation and economic status 
disaggregates are most important when determining the 
underlying causes holding students back from completing 
their enrolled courses. Moreover, the submetrics suggested, in 
addition to the ratio, help further mine the data for populations 
that may require additional assistance and for gaps in course- 
taking and completion patterns, which can be improved by 
institutional intervention through early warning and other 

advising systems. For 
policymakers, both the 
credit completion and 
accumulation metrics 
are primary tools to show academic progression and help 
design and shape policy and funding decisions, and are incor- 
porated in some Outcomes Based Funding (OBF) formulas . 9 


initiatives measure Credit Completion Ratio 


The population and disaggregates counted in this metric 
match those of the framework’s proposed completion metrics 
to maintain consistency across the framework and with the 
field convergence around the four major cohorts. As a result, 
the primary differences between the framework’s definition 
and CCA’s are that this framework disaggregates by creden- 
tial level instead of institution type, is based on full-year cohorts 
instead of fall only, and disaggregates transfer students by full- 
and part-time status. Flowever, full- and part-time students 
could be aggregated when interpreting the metric, without 
confounding results. Because the metric includes all students 
enrolled throughout the year, reporting lag times are expected, 
as spring entrants should receive a full year to complete 
credits. 
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PROGRESSION, continued 


Gateway Course Completion 

Definition 

The percentage of students completing college-level, 
introductory math and English courses tracked separately in 
their first year. Prior credits from AR IB, dual enrollment, 
transfer, and College Level Examination Program (CLEP) do 
count. Credit is earned based on institutional standards. 

Population 

Twelve-month incoming cohorts by credential level and student 
enrollment status and attendance intensity (e.g., FTFT, FTPT, 

TFT, TPT) 

Disaggregates 

Academic preparation, economic status (at entry), race/ 
ethnicity, gender, age, first-generation status, and program of 
study (at entry) 

Submetrics 

• Enrollment in prerequisite remedial courses by subject 

for further 

(if applicable) 

analysis 

• Completion of prerequisite remedial courses by subject 
(if applicable) 


Use Cases 

Gateway course completion in the first year is a key momentum 
point that predicts student success, and the proportion of 
students meeting this momentum point indicates to an institu- 
tion whether students began their college careers on the right 
track . 11 In one respect, it is the best measure of true college 
readiness. For this reason, policymakers too must be keenly 
aware of and aim to use this and other completion measures 
when they are designing and shaping programming, policy, 
and funding. Performance on this metric also can inform insti- 
tutional efforts to help students build academic momentum 
early through counseling and technology-enabled advisory 
systems. 


• Enrollment in gateway courses by subject 

• Number of attempts to complete gateway courses by subject 
•Average time to complete gateway courses by subject 

• Completion of both gateway courses 

•Availability of remedial and gateway courses in sequence 

• Percentages of D’s, F’s, W’s, I’s in gateway courses by 
subject 

• Course engagement/interaction by gateway course 
completion 


Field Usage and Convergence 

Eight voluntary initiatives 10 use gateway course completion 
metrics, though the intricacies of the metric vary by initiative. 
Because no national standards exist for classifying gateway 
courses, individual institutions should define which courses 
count as “gateways,” broadly defined as nonremedial entry- 
level or introductory courses in the subject area. 


Academic preparation is the most critical correlate of gateway 
course completion, and institution leaders can use this disag- 
gregate to evaluate whether preparation or another factor is the 
primary roadblock to on-time gateway completion. Also, age 
and economic status are important disaggregates when 
analyzing these data. Age may demonstrate adverse effects 
related to delayed entry, and economic status may show the 
extent to which lack of funds can delay timely course enrollment 

and completion. 
The recommended 
submetrics can 
further assist insti- 
tutions, as institutions then can understand how remedial 
course-taking, course sequence availability, and time and 
number of attempts to completion can affect this metric and, 
ultimately, student completion. 


• Course format/modality by gateway 
course completion 




initiatives measure Gateway Course Completion 


Similar to the credit completion ratio, the passing grade 
recommendation follows CCA definitions, which includes A, B, 
C, and P grades, in addition to D’s if recognized by the institu- 
tion. The field varies in the time frame used for this metric. CCA 
and VFA measure the percentage of students completing 
gateway courses after one and two years and just two years, 
respectively, while Completion by Design (CBD) measures 
completion after only one year. Because early gateway course 
completion is essential to on-time progression, this framework 
recommends following CBD’s guidelines to signal to the field 
that one year is an important time frame for institutions to 
target for getting students through these courses. 
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Program of Study Selection 

Definition 

The percentage of students in a cohort who demonstrate a 
program of study selection by taking nine credits (or three 
courses) in a meta-major in the first year. Meta-majors include: 
education; arts and humanities; social and behavioral sciences 
and human services; science, technology, engineering, and 
math; business and communication; health; trades 12 

Population 

Twelve-month incoming cohorts by credential level and student 
enrollment and attendance intensity (e.g., FTFT, FTPT, TFT, 

TPT) 

Disaggregates 

Academic preparation, economic status (at entry), race/ 
ethnicity, gender, age, first-generation status, and program of 
study (at entry) 


Submetrics 
for further 
analysis 


• Percentage of students undeclared at entry 

• Number of major changes 


Use Cases 

Concentration in a program of study is an early indicator of 
student progression through higher education, offering more 
information about how and why some students falter. If an 
institution finds that large proportions of their students are not 
concentrating in a major early in their collegiate careers, then 
the institution can adjust advising, course registration, and 
scheduling practices to encourage students to concentrate 
earlier and build momentum toward their degrees. While not 
specifically geared toward students for consumer purposes, 
program of study selection is another progression metric that 
can enhance academic advising and course selection. 


• Likelihood of meeting requirements for entry to intended 
major 


Field Usage and Convergence 

Research by the Community College Research Center (CCRC) 
shows that community college students who do not enter a 
program of study within their first year are much less likely to 
persist and achieve a successful outcome. However, CCRC 
also finds that using students’ declared major or program of 
study is not as useful as using students’ course-taking behavior 
in identifying whether they have successfully entered a 
program of study. As a result of these findings, CCRC recom- 
mends the methodology employed by CBD, in which students 
must complete nine credit hours or three courses in a program 
of study to count as “concentrators” in a major field . 13 The 
framework supports this practice by measuring whether 
students select a program of study within the first year of 
enrollment, regardless of credential level. This recommenda- 
tion diverges slightly from CCA’s practice of measuring major 
selection at entry for all students; however, this framework 
recommends measuring major selection at the end of the first 
year to allow institutions to use the CBD methodology. 


The submetrics are designed to highlight the likelihood of a 
student progressing as well as factors that could contribute to 
slower progression. Specifically, if students change majors 

frequently, an insti- 
tution may need to 
provide more inten- 
sive advising and 
earlier information 
about major tracks and associated career opportunities. Simi- 
larly, if students do not accumulate the credits or GPAs neces- 
sary to enter specific majors, the institution can revisit its 
policies and communication strategies for alerting students to 
the steps they need to take to prepare for their major of interest. 
Detailed degree maps by major are another resource for 
ensuring that students are on a pathway to postsecondary 
success, as demonstrated through research by CCA, CCRC, 
and the Education Trust on guided pathways, Integrated Plan- 
ning and Advising Services (IPAS), and using data to support 
at-risk students . 15 


•Availability of intended major (e.g., wait lists) 
•Availability of detailed degree maps for intended major 
•Availability of prerequisite courses in sequence 



for intended major 

t 


initiatives measure Program of Study Selection 


Ll 



The framework leans on CCA’s classification of meta-majors 
(see list above) because when students enroll early in one of 
these categories, they are able to get on a “guided pathway to 
success .” 14 The meta-majors offer a level of specificity without 
limiting future adjustments to students’ more refined major 
selections. CCA has mapped two-digit Classification of Instruc- 
tional Programs (CIP) codes to the seven meta-majors to allow 
for easy classification and more in-depth analysis. 
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Retention Rate 

Definition 

The percentage of students in a cohort who are either enrolled 
at their initial institution or transfer to a longer program at the 
initial or subsequent institution, calculated annually up to 200 
percent of program length 

Population 

Twelve-month incoming student cohorts by credential level and 
student enrollment status and attendance intensity (e.g., FTFT, 
FTPT, TFT, TPT) 

Disaggregates 

Academic preparation, economic status (at entry), race/ 
ethnicity, gender, age, first-generation status, and program of 
study (at entry) 


Submetrics 
for further 
analysis 


•Timely registration for classes 
• Term-to-term retention rates 

•Retention with advanced class standing (e.g., credits) 


• Stopout or consecutive enrollment rates 


•Academic standing (e.g., GPA, credits) on transfer or dropout 


• Number of credits and degree conferral at transfer out 


Near completion (e.g., fewer than 15 credits) on transfer or 
dropout 

Major/degree at subsequent institution compared with initial 
institution 


rate but are captured in the numerator of the persistence rate. 
Finally, some initiatives, like the VFA, measure retention rates 
term-to-term in the first year, rather than annually from the first 
to second year. To manage reporting burden, this framework 
opts for longer term reporting (up to 200 percent of program 
time) over more frequent (term-based) reporting, although 
institutions could supplement this retention metric by reviewing 
retention data each term. 

Use Cases 

Measuring retention across years enables an institution to 
decipher when and which students stopout and dropout and, 
through subsequent investigation of submetrics, determine 
why. For example, a student dropping out after one year is 
very different from a student dropping out just short of a 
credential. Parsing the different times for stopout and dropout, 
especially for different student populations such as underrep- 
resented minorities, allows institutions to target interventions 
to address students’ specific barriers or needs. 


•Withdrawal rate (percentage of all enrolled 
students who leave in one year) 




initiatives measure Retention Rate 


Field Usage and Convergence 

First-year retention rates are a commonly used metric of 
student progression, collected in IPEDS and sixteen voluntary 
initiatives reviewed for this project. 16 However, retention rate 
specifications often do not align with other progression and 
outcome metrics, such as graduation rates. This framework’s 
retention definition aims to leverage the field’s best thinking 
about progression and completion to design a retention metric 
that parallels those efforts. 


Programs and institutions can 
further use the recommended 
submetric data on academic 
momentum and achievement at the time of stopout or dropout 
to better understand if students are leaving their institution due 
to income constraints, low achievement, or alternative reasons. 
While largely used as an institutional improvement measure, 
retention rates can also serve as important signals for both 
prospective students, who can use retention to select institu- 
tions where they have the best chance of persisting, and poli- 
cymakers, who can design policies and programs that promote 
higher retention rates. 


For instance, CCA and IPEDS use fall enrollment cohorts to 
measure retention, but this framework proposes a 12-month 
cohort to capture more nontraditional students. Also, while 
IPEDS and other commonly used retention metrics measure 
retention only after one year, this framework proposes tracking 
retention every year up until 200 percent of program length to 
provide continuous updates on the progress of each cohort. 
Additionally, while VFA does not count transfer students as 
retained and CCA includes in the numerator of the retention 
rate students who transfer to any level of institution or program, 
this proposed measure includes in the numerator only students 
who transfer to a longer program. This recommendation aligns 
with the framework’s proposed retention rate with the proposed 
outcome measures (graduation, transfer, and success). 
Students who transfer into degree programs with the same or 
shorter length do not count in the numerator of this retention 
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Persistence Rate 

Definition 

The percentage of students in a cohort remaining enrolled or 
earning a credential at their initial or subsequent institution, 
measured annually up to 200 percent of program length 

Population 

Twelve-month incoming student cohorts by credential level and 
student enrollment status and attendance intensity (e.g., FTFT, 
FTPT, TFT, TPT) 

Disaggregates 

Academic preparation, economic status (at entry), race/ 
ethnicity, gender, age, first-generation status, and program of 
study (at entry) 

Submetrics 
for further 
analysis 

• Stopout or consecutive enrollment rates 

•Academic standing (e.g., GPA, credits) on transfer or dropout 

• Number of credits and degree conferral at transfer out 

•Near completion (e.g., fewer than 15 credits) on transfer or 
dropout 

• Major/degree at subsequent institution compared with initial 
institution 

•Withdrawal rate (percentage of all enrolled students who 
leave in one year) 


initiatives measure Persistence Rate 


Field Usage and Convergence 
The persistence metric in this framework is based largely on 
the Student Achievement Measure (SAM), an initiative 
supported by the American Association of Community 
Colleges (AACC), the American Association of State Colleges 
and Universities (AASCU), the Association of American Univer- 
sities (AAU), the American Council on Education (ACE), the 
Association of Public and Land-grant Universities (APLU), and 
the National Association of Independent Colleges and Univer- 
sities (NAICU). 

Specifically, the framework’s recommendations closely mirror 
SAM’s bachelor’s-seekers model. For instance, this frame- 
work recommends including enrollment and completion at 
subsequent institutions in the persistence rate numerator 
because those figures can provide useful feedback to all 
colleges, particularly two-year colleges, to help support 
students along their degree pathways. Thus, the numerator of 
the persistence metric includes counting students who: have 
earned any credential at their initial institution, have earned 
any credential at any subsequent institution, are still enrolled 
at the initial institution, or are still enrolled at any subsequent 
institution. This methodology matches SAM’s bachelor’s- 
seeking model, but differs from its associate’s-seeking model, 
which does not require institutions to report completions at 
subsequent institutions and reports only once at the end of the 
six-year reporting period. By applying consistent metric defini- 
tions to all credential levels, this framework aims for results 
that are more comparable across institutions. Additionally, the 
associate’s-seeking model for SAM is scheduled to expand 
this fall to offer an option of applying the bachelor’s-seeking 
model to associate’s-seeking students. 


This framework’s proposed persistence metric does expand 
on SAM by using 12-month cohorts for all three undergrad- 
uate credential levels (bachelor’s degree, associate’s degree, 
certificate) and all four incoming student cohorts (FTFT, FTPT, 
TFT, TPT) as detailed in the cohort specifications section of 
this paper, instead of the fall-start cohorts SAM uses, as a 
reflection of IPEDS definitions. As currently specified, SAM’s 
bachelor’s-seeking model requires only two cohorts: FTFT 
and TFT, although institutions may opt to report FTPT and TPT 
students also. SAM also reports outcomes to only 1 50 percent 
of program time, whereas some collections, like the VFA and 
IPEDS Outcome Measures, track certificate- and associate’s- 
seeking students to 300 percent or 400 percent of program 
time, stating that this additional time could capture additional 
students who may have stopped out or dropped out during 
their education careers. 17 This framework strikes a compro- 
mise, proposing that persis- 
tence rates be measured 
annually up to 200 percent of 
program time, signaling the 
importance of timely completion while also allowing some flex- 
ibility for students who take longer to complete. 

The framework does distinguish this persistence metric from 
the proposed success metric, which measures whether 
students earn the credential sought at their initial institution or 
transfer to a longer degree program at the initial or subsequent 
institution, because the field is using both of these measures 
for complementary, yet distinct uses. This measure is designed 
to include those students not captured in the numerator of the 
retention or success measure, for a more comprehensive view 
of student persistence. By tracking all transfers and comple- 
tions, the student-centric persistence metric is designed to 
present a comprehensive picture of student movement 
throughout the postsecondary system, while the institution- 
centric success metric is designed to focus colleges and 
universities specifically on their institutional contributions to 
students’ outcomes (for more detail, see the section on 
outcome rates). 

Use Cases 

Along with retention and outcome rates, institutions, prospec- 
tive students, and policymakers can use persistence rates to 
better understand the full range of outcomes for college 
students. For institutions, for instance, the persistence rate 
signals a credible target for improving their success rates, 
because students who are persisting elsewhere might have 
graduated from their initial institution instead. The persistence 
rate is also useful for institutions that aim to prepare many of 
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their students for transfer, so they can demonstrate their prog- 
ress and success. 

Institutions can also use information gleaned from the full 
range of student progression outcomes reflected in the persis- 
tence rate to inform student support policies and programs, as 
well as academic programs. The submetrics mirror those 
recommended for retention and can help institutions evaluate 
when, which, and why students move within and outside of the 
institution. Policymakers can use persistence rates to evaluate 
how students progress through the higher education system, 
especially when examined within a state. Finally, students can 
use persistence rates in concert with outcome rates to under- 
stand how they may fare by beginning at a particular institu- 
tion, while also accounting for potential success elsewhere. 
These rates add another layer of information that prospective 
students and families can use to make the best and most 
informed higher education decisions. 
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COMPLETION 


Outcome Rates: Transfer Rate, Graduation Rate, and Success Rate 


Definition Transfer rate: The percentage of students in a cohort who 

transfer into longer programs at the initial or subsequent 
institution(s), up to 200 percent of program length 

Graduation rate: The percentage of students in a cohort who 
earn the credential sought at their initial institution, up to 200 
percent of program length 

Success rate (graduation rate + transfer rate): The percentage 
of students in a cohort who either graduate with the credential 
initially sought at the initial institution or transfer to a longer 
program at the initial or subsequent institution (s), up to 200 
percent of program length 

Each outcome rate should be captured at least at 100 percent, 
150 percent, and 200 percent of program length, and should 
be reported in real-time, not retroactively. 18 


Population 


Twelve-month incoming student cohorts by credential level 
sought, enrollment status, and attendance intensity at entry 
(e.g., FTFT, FTPT, TFT, TPT) 


Disaggregates Academic preparation, economic status (at entry), race/ 

ethnicity, gender, age, first-generation status, and program of 
study (at entry) 


Submetrics 
for further 
analysis 


• Stopout or consecutive enrollment rates 

• Graduation rates by number of transfer-in credits (if applicable) 
•Academic standing (e.g., GPA, credits) on transfer or dropout 

• Number of credits and degree conferral at transfer out 

•Near completion (e.g., fewer than 15 credits) on transfer or 
dropout 

• Major/degree at subsequent institution compared with initial 
institution 


•Withdrawal rate (percentage of all enrolled 
students who leave in one year) 




Field Usage and Convergence 
More accurately accounting for the success of all 
students has been a top priority of many volun- 
tary initiatives over the past decade. In fact, 1 8 of 
the initiatives reviewed for this framework collect 
some variation on the proposed metrics, with varying levels of 
detail. IPEDS, SAM, VFA, Access to Success (A2S), and CCA 
in particular provide the foundation for this framework’s 
proposed outcome measures, disaggregates, and calcula- 
tions. Reflecting the advancements in the field, this framework 
recommends tracking these outcomes for four 12-month 
enrollment cohorts. The four cohorts (FTFT, FTPT, TFT, TPT) 
are based on the cohorts defined in the IPEDS OM survey as 
well as the voluntary initiatives. Precedent also exists in the 
field for tracking separate cohorts based on credential sought, 
as the federal government now requires institutions to report 
credential-seeking status by credential level for federal aid 
purposes, and CCA and SAM report separate cohorts based 
on credential level sought. Although some students will 
change credential levels throughout the course of their studies, 
most students do not. Of those students who attain a degree, 


Beginning Postsecondary Student (BPS) Survey shows that at 
least 85 percent receive their initially sought degree at the first 
institution attended, as opposed to a subsequent institution. 19 
Similarly, according to a recent National Student Clearing- 
house (NSC) report, 77 percent of the 2009 cohort that 
completed a degree did so at the initial institution. 20 

Finally, in terms of transfer, CCA tracks transfer only from a 
two-year to a four-year institution, while IPEDS OM and SAM 
report all transfers combined. This framework builds and 
expands on this previous work, recommending that institu- 
tions report transfer from a shorter to a longer credential 
program separately from transfer to a credential program of 
shorter or the same length. Only transfer to a longer program 
is counted in the recommended success rate, but all types of 
transfer are captured in the persistence metric described 
earlier. To better reflect student pathways, CCA recently began 
collecting a success rate, using the same calculation as 
recommended by the framework. This differentiation of trans- 
fers by level of receiving credential program in the success 
rate is particularly relevant for measuring the success of 
community-college students seeking transfer to four-year 
programs. New research also stresses the importance of 
tracking transfer from community college to four-year institu- 
tions to measure the effective- 
ness of an institution’s ability to 
support students through the 
transfer process. 21 Some initia- 
tives have reported in the past 
that it is not possible to reliably 
identify the length of the 
transfer credential program, 
only the level of the transfer 
institution. However, due to 
new federal reporting requirements for loan and Pell eligibility, 
the Clearinghouse is reporting near complete coverage on 
credential length in 2015 in its enrollment file, so we are confi- 
dent that institutions can make this distinction or should be 
able to in the near term. 

Use Cases 

These outcome rates provide a more complete picture of how 
effectively students achieve their postsecondary objectives, 
highlight institution-level student success, and best reflect the 
information needed by students, policymakers, and institu- 
tions to understand and improve student outcomes. Outcome 
rates are used in tandem with persistence and retention rates 
to explore student mobility and success in higher education 
even more fully. 22 


1 7 ) initiatives measure Transfer Rate 


18 ) initiatives measure Graduation Rate 


3 j initiatives measure Success Rate 
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Disaggregated for student characteristics like race/ethnicity, 
economic status, age, first-generation status, and academic 
preparation, outcome rates can be particularly useful in 
helping institutions target their efforts to promote equitable 
results among all of their students. For example, examination 
of the disaggregated outcome rates recommended here could 
allow institutions to identify groups of students that need more 
support and subsequently design interventions to move more 
of those students toward completion. In particular, nearly every 
initiative examined disaggregates completion rates by Pell 
status, demonstrating that such reporting is feasible and useful 
for institutions and policymakers — both of whom are increas- 
ingly interested in the success of low-income students. 
Furthermore, understanding what types of programs students 
transfer to can give colleges insight into whether they are 
helping students reach their next intended degree goal or 
whether they can improve by retaining students who are 
simply leaving to attend other institutions. Given that 
completing a credential is the primary goal of most students, it 
is crucial that institutions take a close and frequent look at the 
completion outcomes of all of their students. 
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Completers 

Definition 

The number of students who complete a credential in a given 
year 

Population 

All completers in a given year by credential level attained 

Disaggregates 

Race/ethnicity, gender, age, academic preparation (at any 
time), economic status (at any time), first-generation status, 
program of study (at exit), and part-time (at any time) and 
transfer status 

Submetrics 

• Crosstabulations of credentials awarded by key 

for further 

disaggregates (e.g., race and gender) 

analysis 

• Distribution of credentials awarded by program of study 


• Distribution of credential awarded to underrepresented 
populations 

• Credentials awarded to underrepresented populations in 
STEM 


Field Usage and Convergence 

This completers metrics recommends counting the number of 
students who complete, as opposed to the number of creden- 
tials completed. This specification follows convention for the 
new completers measure added to IPEDS in 2011-12. While 
IPEDS collects counts of both completers (number of students) 
and completions (number of degrees/certificates), this frame- 
work recommends using completers as the primary metric, as 
it aligns with national goals to raise attainment by graduating 
more credential completers. However, it may still be appro- 
priate to track and report completions (including students who 
earn multiple degrees at the same time) for other purposes. 


Use Cases 

Institutions can use counts of completing students to demon- 
strate productivity and their institutional contribution to the 
workforce and society. Especially when disaggregating by 
demographic characteristics, top-performing institutions can 
make the case that they are contributing large numbers of 
underrepresented college graduates. Alternately, these data 
on completers could show that some institutions are producing 
very few graduates in certain fields (e.g., STEM) or from certain 
student groups (e.g., African Americans) or a cross between 
the two (e.g., African American STEM graduates). These 
results can trigger the college to investigate the cause for 
small numbers or gaps and evaluate whether their credential 

awarding patterns align with insti- 
tutional goals and workforce 
needs. Students and policymakers 
can employ this metric to examine 
the types of students that succeed at a particular college, 
contributing to informed school selection and strategic poli- 
cies that advance those institutions that serve all students well. 
For example, many states include the number of credentials 
awarded or students completing — particularly for underrepre- 
sented student groups — in their outcomes-based funding 
formulas. 24 


•Time and credits to credential 




initiatives measure Completers 


IPEDS disaggregates the number of completers by credential 
level, gender, race/ethnicity, and age and the number of 
completions by credential level, program of study, race/ 
ethnicity, and gender. This framework recommends using all 
of these disaggregates already in IPEDS and adding economic 
status, academic preparation, first-generation status, and part- 
time and transfer status as recommended disaggregates. In 
addition to IPEDS, the VFA and others 23 collect unduplicated 
completion counts, which are functionally similar to this 
proposed completers metric. 
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Time and Credits to Credential* 

Definition 

Time to credential: The average time accumulated from first 
date of entry to the institution to date of completion for all 
completers in a given year 

Credits to credential: The average credits accumulated from 
first date of entry to the institution to date of completion for all 
completers in a given year 

Population 

All completers in a given year by credential level attained 

Disaggregates 

Race/ethnicity, gender, age, academic preparation (at any 
time), economic status (at any time), first-generation status, 
program of study (at exit), and part-time (at any time) and 
transfer status 


Submetrics 
for further 
analysis 


• Average number and percentage of transfer credits accepted 
(if applicable) 

• Number of course D’s, F’s, W’s, I’s or retakes 

• Major declaration/major changes 

• Stopout or continuous enrollment rates 

• Cumulative debt by time or credits to credential ^ 


Use Cases 

This metric allows institutions to analyze how efficiently 
students complete credentials, flagging potential inefficiencies 
to be addressed. First, institution and department leaders can 
use these data to understand which programs take longer to 
complete and thus may be more costly options for students, 
as well as programs that need curricular review to determine if 
degree requirements are set appropriately. Some credential 
requirements may be outdated and could be streamlined to 
reduce the number of credits required for completion while still 
maintaining quality. In some cases, students may be taking 
unnecessary courses because credential pathways are not 
communicated clearly or because the courses they need for 
their credential are unavailable, which can be addressed in the 
academic advising and scheduling process. 


initiatives measure Time and Credits to Credential 


‘Although classified as an efficiency metric in the framework, 
this is presented in this chapter because of its close relationship 
with the completers metric. 


Field Usage and Convergence 

Time and credits to credential measure the efficiency with 
which students complete their degrees or certificates. As such, 
it is classified as an efficiency metric in this framework but 
presented here alongside the performance metrics because it 
is so closely connected to the completers metric. Time and 
credits to credential have become commonplace in the field, 
with eight initiatives, including ATD, CCA, and the PAR Frame- 
work calculating similar metrics . 25 

This metric measures only credits accumulated and time spent 
at the specific institution of interest, though if measuring for a 
state or system, credits at any institution in that state or system 
should be included. The framework proposes using CCA’s 
definitions and methodology for remedial courses and stop- 
outs. For example, remedial courses should count toward the 
total accumulations, regardless of whether the credits count 
toward degree completion, to provide a comprehensive 
picture of time and credit accumulation for students at varying 
levels of academic preparation . 26 To control for outliers and 
reflect institutional practices related to stopouts, this recom- 
mended calculation excludes students who stopout for more 
than five years . 27 As for disaggregations, the framework builds 
on those required by CCA to further align these metrics with 
many of the others included in the framework. These disag- 
gregates also allow for deeper and more dynamic analysis. 


Also, if certain 
student popula- 
tions tend to take 

more courses than needed or take a long time to complete, 
corrected pathways and additional supports can be imple- 
mented at the college or department level to intervene with 
additional advising for students at risk of extended time to 
credential. Additionally, those institutions with favorable 
transfer policies should show lower rates of time and credits to 
degree because acceptance of transfer credits enhances effi- 
ciency. In cases where the opposite is true, transfer policies 
could be reevaluated to decrease time to credential. Students 
also can use these data to inform college decision-making. 
Because time and credits to credential directly affect college 
affordability for many students, knowing these outcomes 
manages expectations for personal finance and time that 
should be dedicated to higher education. For policymakers, 
longer-than-average time to completion can signal inefficient 
use of federal or state funds. 
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Net Price* 

Definition 

The average cost of attendance (COA) for an institution less all 
grant aid in a given year 

Net Price = COA - All Grant Aid 

COA follows federal definitions for costs associated with a year 
of college, including tuition and fees; room and board 
(determined by living arrangements); books and supplies; and 
other expenses, like travel and personal items 

Grant aid includes grants from all sources (federal, state or 
local, institutional, and other) 

Population 

FTFT, and all full-time undergraduates by credential level; 
includes all students, not just aid recipients; excludes 
out-of-state students 

Disaggregates 

Credential level, economic status (at that time), academic 
preparation, race/ethnicity, gender, age, first-generation status, 
program of study (at that time) 


Submetrics 
for further 
analysis 


• Percentage of students applying for aid 

• Percentage of students receiving grant aid (by type or source) 

• Net price for students receiving grant aid 


• Net price by dependency status 


• Net price divided by average income within quintiles 


• Net price for part-time, transfer, out-of-state students 


• Net price by year in college 


• Number of hours worked 


the net price figure is often displayed separately from the 
percentage of students receiving aid, creating a misleadingly 
low result by omitting full-pay students. Because of this limita- 
tion, the framework recommends including nonaided students 
in the calculation to present an “upper limit” of what students 
might be expected to pay. 28 

Others have noted that income data may not be available for 
non-Title IV recipients, making it difficult to disaggregate net 
price by income if all students are included. Although income 
data will be missing for some students not receiving federal 
aid, it will be available for all FAFSA filers, accounting for 70 
percent of students, even if they do not receive aid, because 
that information is provided back to institutions. Indeed, for 
students whose family income falls in the upper two quintiles, 
at least 55 percent of students apply for federal aid. 29 Also, 
some colleges collect income information through their own 
financial aid applications or other methods, supplementing the 
FAFSA data. Therefore, it is recommended that institutions use 
all available income data and classify remaining students in an 
income-unknown category. 


• Number of dependents 


*Also see the related unmet need metric. 




initiatives measure Net Price 


Field Usage and Convergence 

This net price metric recommendation is based on the current 
IPEDS collection but expands on it by including non-grant aid 
or non-Title IV aid recipients, adding a cohort for all full-time 
undergraduates and cohorts by credential level, allowing for 
analysis of grant aid by source, and adjusting the income 
bands. This metric follows IPEDS methodology by calculating 
a weighted average COA based on students’ living arrange- 
ments (on-campus, off-campus with family, off-campus not 
with family). COA for non-Free Application for Federal Student 
Aid (FAFSA) filers may need to be estimated if living arrange- 
ment information is not available through other sources. 

Including nonarant/non-Title IV aid recipients 
The precedent in IPEDS is to include only first-time, full-time 
grant/Title IV aid recipients in net price reporting, but this 
framework recommends expanding coverage by adding 
nongrant/non-Title IV aid recipients into the cohorts to surmise 
a more accurate picture of net price for more of the student 
population. Some contend that displaying the net price for 
aided students next to the percentage of students receiving 
aid provides a comprehensive enough picture of net price for 
all students. While these two data points together are valuable, 


Adding a cohort for all full-time 
undergraduates and separate cohorts 

for each credential level 
This framework also recommends adding a cohort of all full- 
time undergrads to supplement the first-time, full-time net 
price cohort. This all-undergraduate cohort will capture pricing 
for continuing and transfer students who may receive different 
levels of aid as compared with first-time students. While the 
all-undergraduate net price cohort would be an addition to 
IPEDS, including it aligns with other portions of IPEDS that 
collect a measure for both a FTFT cohort and an all under- 
graduate cohort (e.g., number and size of Pell Grant awards 
and federal student loans). Also, the framework expands on 
IPEDS by recommending separate net price calculations for 
each credential level offered at the institution. IPEDS data 
currently group students together regardless of credential 
level sought, but COA and grant aid may vary by credential 
level. While the framework includes only full time, in-state 
students to normalize costs, part-time and out-of-state student 
net prices could be calculated separately as submetrics to 
better understand the financial situations of those populations. 

Allowing for analysis of grant aid bv source 
While not impacting the ultimate net price figure, this frame- 
work recommends reporting the federal, state, and institu- 
tional grant aid separately for each income level — rather than 
combined, as IPEDS does now. Reporting separate amounts 
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for each type of grant aid will allow one to evaluate the distribu- 
tion of grant aid across income levels for each source. These 
measures of grant aid by source already are used to calculate 
the IPEDS net price figure, but they are not reported sepa- 
rately, confounding the impacts of federal, state, and institu- 
tional financial aid policies on students of varying income 
levels. However, average federal, state, and institutional grant 
aid are available elsewhere in IPEDS, just not disaggregated 
by income. These data would be more valuable if disaggre- 
gated by income level and better integrated across surveys. 

Changing the income bands 

Finally, this framework advances recommendations for new 
net price income bands using quintiles based on the annually 
published American Community Survey (ACS) family income 
data . 30 As the one-year ACS survey estimates are published 
two months ahead of the IPEDS net price data collection, the 
income categories can be updated annually to better reflect 
the landscape of families and students applying for aid. While 
the income thresholds will change annually, the methodology 
will offer consistency by always representing national income 
quintiles. These income quintiles demonstrate a change from 
the current IPEDS net price income categories, the origins of 
which are unclear, although they appear to approximate ACS 
quintiles from the time the net price legislation was being 
drafted in Congress . 31 They are not indexed to change over 
time, although the Secretary of Education has the authority to 
adjust them. 

Use Cases 

This metric is likely best known as a consumer information 
tool, providing students and families with the likely cost of 
higher education at any given institution based on their income 
level. However, current net price figures can be misleading 


because they apply only to students who receive aid, thus 
omitting the amount paid by nonaided students and making 
the results particularly unrepresentative of higher-income 
students who are less likely to receive Title IV aid . 32 Because 
students do not know whether they will receive aid before 
applying, they do not know if the IPEDS net price figures will 
apply to them. This framework’s proposed metric provides 
comprehensive net price data for all consumers to evaluate 
expected prices and for policymakers to assess college afford- 
ability for all attendees, not just recipients of federal funding. 
Policymakers also should use net price results to evaluate how 
institutions and states spend their aid dollars and determine 
whether their practices align with the priorities of the federal 
government in lowering the net price for low-income students. 
Regardless of the adoption of this proposed net price metric, 
institutions should improve and better publicize their net price 
calculators so prospective students and families can obtain a 
more customized estimate of their expected price . 33 

For institutional improvement, financial aid officers and other 
college administrators can use these more inclusive figures to 
evaluate how much they are expecting students from different 
income levels to pay and can adjust financial aid policies and 
target intervention strategies accordingly. If, for instance, the 
net price is higher for low-income students than high-income 
students, the institution or state should redistribute grant aid 
toward the students with greater need. The net price submet- 
rics also can be especially useful to guide institutional action. 
For example, if an institution finds that net price increases 
substantially for low-income students based on their year in 
program, they can reevaluate policies to implement more 
predictable prices across time. 
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Unmet Neec 

i* 

Definition 

The average net price for an institution less the average 
expected family contribution (EFC) in a given year 

CO A - All Grant Aid - EFC = Net Price - EFC 

Population 

FTFT, and all full-time, undergraduates by credential level; 
includes all students, not just aid recipients; excludes 
out-of-state students 

Disaggregates 

Credential level, economic status (at that time), academic 
preparation, race/ethnicity, gender, age, first-generation status, 
program of study (at that time) 


Submetrics 
for further 
analysis 


• Percentage of students applying for aid 

• Percentage of students receiving aid 

• Percentage of students with unmet need and their average 
unmet need 

• Unmet need for aid recipients by type or source 

• Unmet need by year in college 

• Part-time, transfer, and out-of-state unmet need 

• Student payment methods for meeting unmet need 

• Completion rates by level of unmet need 

• Number of hours worked 

•Number of dependents ^ 


‘Also see the related net price metric. 


Field Usage and Convergence 

This unmet need metric expands on what is currently collected 
through the IPEDS net price metric, by incorporating EFC into 
the calculation. Unmet need is used frequently by institutions, 
advocates, and researchers to evaluate the adequacy of finan- 
cial aid in meeting students’ financial needs . 34 However, an 
institution may not have data to calculate unmet need for all 
students in the cohort because of missing EFC information, 
which is calculated via the FAFSA. The framework recom- 
mends that reporting strive to be as complete as possible, 
given available data. Unmet need data are included in the 
National Postsecondary Student Aid Survey but are reported 
only at the national — not the institutional — level, which is prob- 
lematic for all key constituencies needing the data. 


Use Cases 

As part of financial aid awarding processes, institutional 
leaders should use unmet need to drive organizational change 
and improvement around equitable access to higher educa- 
tion. Accounting for EFC through an unmet need calculation 
offers a better understanding of the financial hurdles facing 
students of varying income levels and allows institutions to 
promote practices and processes that are mindful of afford- 
ability for all students. If, for example, the institution finds 
overmet need (negative unmet need) for high-income groups 
and substantial unmet need among low-income groups, which 
is the case in nationally representative survey data, it can 
adjust its financial aid policies to redirect aid to the students 
with remaining financial need. 

The submetrics help to clarify how financial aid application, aid 
received, payment methods, dependency status, and work 
burden can impact unmet need, and a student’s ultimate 
ability to pay for college. While much of these data are acces- 
sible through the National Center 
on Education Statistics’ (NCES) 
sample studies, they are not avail- 
able at the institution level, so the 
collection and analysis of these data could fill a large gap in 
the field’s understanding of the variability of unmet need 
across colleges. 

Students can use unmet need to evaluate whether that partic- 
ular institution is affordable for them and how it financially 
serves students in similar financial situations. Policymakers 
could use this metric in tandem with net price to assess the full 
scope of financial burden that is placed on students and fami- 
lies and adjust financial aid policies accordingly — or encourage 
institutions to do so. 


initiatives measure Unmet Need 


While some prefer to use net price or unmet need, the frame- 
work encourages the field to consider both metrics in conjunc- 
tion with each other, as they help understand the price of 
higher education in relation to family or personal income in 
different ways. For example, net price can be calculated as a 
percentage of family income, as a measure of affordability, 
whereas unmet need provides a concrete dollar amount that 
students must find a way to finance, above and beyond what 
their family can afford to pay. 
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Cumulative Debt 

Definition 

The median amount of debt student borrowers incur while 
attending an institution or program. Includes all sources of 
student debt — federal, state, institutional, and private loans 

Population 

All undergraduate borrowers who leave the institution in a 
given year (completers and noncompleters, but disaggregated) 

Disaggregates 

Credential level, completion status, economic status (at any 
time), enrollment status, attendance intensity (at any time), 
program of study (at exit), race/ethnicity, academic preparation 
(at any time), age, gender, first-generation status 


Submetrics 
for further 
analysis 


• Percentage of students borrowing overall and by type of loan 

• Loan debt by type of loan 

• Loan debt by dependency status 

• Cumulative loan debt across all institutions 
attended for transfer students (if available) 


initiatives measure Cumulative Debt 


Field Usage and Convergence 

The College Scorecard uses median cumulative loan debt as 
a core debt measure, derived from National Student Loan 
Data System (NSLDS) data. 35 The voluntary Common Data Set 
also collects debt data, which have served as the basis for 
most analyses of student debt to date. 36 This framework’s 
proposed definition for median debt is similar to the College 
Scorecard standards in many ways, as discussed below. Key 
components of this metric design include using completion 
status as an essential disaggregate, reporting the percentage 
of students borrowing as an essential submetric, and counting 
all debt accumulated at the reporting institution, including 
Parent PLUS and private loans. 


and 95 percent of students borrowing is performing very differ- 
ently from an institution with $30,000 in median debt and 5 
percent of students borrowing. This combination creates the 
foundation for understanding loan borrowing patterns for an 
institution, and both should be reported together. 

This methodology differs from the framework’s recommended 
net price definition, which includes all students — aided and 
unaided. Including all students in the net price measure results 
in a higher estimate of what students might be expected to pay 
than if nonaided students were omitted. In the same regard, 
excluding nonborrowers from the median debt calculation 

leads to a higher estimate of 
debt than if nonborrowers 
were included. In both cases, 
the decision reflected in the 
framework is designed to provide students with a sense of the 
greatest likely financial risk. 

Count all debt accumulated at the reporting institution, including 

Parent PLUS and private loans 

While the Scorecard includes only federal student loan debt, 
excluding Parent PLUS and private loans, this framework 
recommends including all loans, including Parent PLUS and 
private loans. While PLUS loans are taken out by a different 
person (parent rather than student), they still contribute to the 
family’s total debt required to pay for college, and combining 
them with student loan debt paints a more complete picture of 
college affordability or lack thereof. 


Completion status is an essential disaggregate 
This framework recommends including all students exiting the 
institution during a given year in the median debt measure, in 
addition to reporting separate cumulative debt figures for 
completers and noncompleters. 37 In other words, completion 
status is an essential disaggregate for this debt measure 
because debt aggregated across completers and noncom- 
pleters could confound results. For example, if a typical 
student leaves a college with a degree and $30,000 in debt, 
that college is serving students far better than a college where 
the typical student leaves with $30,000 in debt and no degree. 
Reporting cumulative debt for noncompleters is necessary as 
well because these students are most likely to struggle with 
repayment. 38 

The percentage of students borrowing is an essential submetric 

To provide context around the accumulated loan debt, the 
framework also strongly recommends reporting the percentage 
of the cohort that borrowed any loans as an essential 
companion metric. An institution with $30,000 in median debt 


In addition to including Parent PLUS loans, the framework also 
recommends including private student loan debt in this metric 
to discern the full extent of borrowing. While not required by 
law, most lenders require that an institution certify student 
enrollment at the time of the loan, so institutions should be 
able to keep records on private borrowing. 39 This private loan 
data collection goes beyond what is reported on the College 
Scorecard, but the benefits of understanding the extent of 
nonfederal loans, which often carry high interest rates and are 
void of the consumer protections federal loans afford, outweigh 
the additional burden of collection. If an institution prefers not 
to rely on its own records, it can access private student loan 
data through a contract with MeasureOne, a third-party that 
captures private student loan volume from the six major 
lenders, representing 71 percent of the student loan market. 40 
One potential drawback to the inclusion of private loans is that 
while institutions are aware of the aid amounts as disbursed 
through the institution, they are not necessarily aware of 
earned interest or payments made while enrolled, which 
impact the total loan debt at exit. Regardless of this limitation, 
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including the initial private loan amounts would still help to 
capture the amount borrowed to cover the cost of the degree. 

While the framework recommends including all loan types in 
the cumulative debt amount, it also recommends disaggre- 
gating the metric by loan type to parse out the impact of 
different loan programs, like Parent PLUS and private loans. 
The framework also recommends disaggregation by creden- 
tial level and program of study because students should be 
able to view debt and earnings side by side, by program, to 
understand their earnings prospects in relation to debt. Addi- 
tionally, emergency short-term loans from institutions are not 
to be included in this metric, as their repayment period 
diverges from other federal loans and are not intended to 
cover unmet need — only emergencies. Finally, this framework 
recommends that the median cumulative loan debt metric 
include only the debt accrued from the reporting institution, as 
the Scorecard does, but the total cumulative debt submetric 
should include the students’ total debt, regardless of institu- 
tion or program, if available. This submetric will provide a more 
comprehensive view of the debt burden students carry into 
their post-college lives. 


Use Cases 

Understanding student loan debt is a necessary component to 
measuring institutional performance for policymakers and 
institutions alike, as financing can impact student access, 
progression, and completion. Specifically for cost metrics, the 
distinction among median debt among students of different 
economic statuses is essential, as high costs limit access to 
low-income students and further stratify higher education. 
With the disaggregates and submetrics, especially specific to 
low- and moderate-income students, institutions can use 
these data to develop better, more targeted counseling and 
services for populations who may be at risk of high student 
loan debt. Institutions and policymakers also can use the 
disaggregated debt data to help craft financial aid policies to 
reduce debt, especially for the most economically vulnerable 
students, as they are more likely to take on loan debt . 41 

Debt data also can be used to inform student decisions in the 
same way as net price, providing prospective students with a 
better understanding of how students in similar situations fare 
at the institution. Median cumulative debt seeks to quantify 
both affordability and financing methods used by typical 
students at each institution. While total loan volume across an 
entire institution, available on the Federal Student Aid Data 
Center, is a useful data point for evaluating broader trends 
regarding student loans, the median cumulative debt better 
demonstrates what is required financially of a typical student. 
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Loan Repayment and Default Rates 

Definition 

Loan repayment rate: The percentage of borrowers in a cohort 
who make at least $1 of progress on their loan principal in a 
fiscal year, measured at one, three, five, and 10 years into 
repayment 

Number of Borrowers Paid in Full + Number of Borrowers in 
Active Repayment 

Number of Borrowers Entering Repayment 

Cohort default rate (CDR- federal three-year rate): The 
percentage of borrowers who enter repayment in a fiscal year 
and default in three fiscal years 42 

Population 

All borrowers who enter repayment in a given year. 

Disaggregates 

Undergraduate versus graduate status, completion status, 
economic status (at any time while enrolled), program of study 
(at exit), race/ethnicity, enrollment status, attendance intensity 
(at any time while enrolled), academic preparation (at any time 
while enrolled), age, gender, first-generation status 

Submetrics 
for further 
analysis 

• Incidence of deferment, forbearance, and delinquency 

• Use of income-driven repayment plans 

•Average amount of defaulted loan 

• Loan repayment and cohort default rates by loan type 

• Student Default Risk Index 


institutions accountable for default on debt taken out to attend 
their institution or previous institutions, institutions that can 
parse out the default rate for loans from their institution alone, 
using their LRDR and SPR, could provide better outreach and 
counseling to students at risk. This framework also recom- 
mends the Institute for College Access and Success’ (TICAS) 
proposed Student Default Risk Indicator (SDRI) as a submetric 
to CDRs. The SDRI is calculated by multiplying the CDR with 
the institution’s borrowing rate to better contextualize the 
borrowing environment of the institution. 47 Either a lower 
borrowing rate or a lower default rate would improve perfor- 
mance on this metric. 

Repayment rates gained prominence during the past five 
years through gainful employment regulations and proposals 
related to federal risk-sharing and accountability; they were 
included in the September 2015 release of the College Score- 
card data. 48 These rates are valuable because borrowers could 


3 ) initiatives measure Loan Repayment and Default Rates 




Field Usage and Convergence 

The framework recommends using CDRs and repayment 
rates in tandem because CDRs measure the worst outcome 
for students — default — and repayment rates complement by 
showing whether student make at least minimal progress 
annually on their loans. CDRs are calculated and released by 
the U.S. Department of Education (ED) annually to determine 
institutional federal financial aid eligibility, and repayment rates 
were released on the latest College Scorecard. 43 The frame- 
work proposes maintenance of current field practice and defi- 
nitions because institutions must work with the data that ED 
provides currently. Despite concerns around gaming CDRs, 44 
this metric fills a crucial role in measuring institutional perfor- 
mance, by showing the frequency of students’ worst debt- 
related outcome. Research has shown that institutions can 
reduce student loan default through targeted actions, improved 
performance on the metric and, more importantly, borrowers’ 
financial situations. 45 Institutions are becoming more creative 
in ways to reach out to delinquent borrowers and help them 
into a positive repayment status before default. 

Federal CDRs currently exclude PLUS, Perkins, consolidation, 
and private loans, although the framework recommends that 
institutions attempt to calculate for these loans default rates 
based on data from their Loan Record Default Reports (LRDR) 
and School Portfolio Reports (SPR) available through the 
Office of Federal Student Aid (FSA). 46 Because CDRs hold 


be avoiding default but experiencing other poor outcomes, 
such as delinquency or negative amortization. The framework 
recommends using the basic calculation and definition param- 
eters used in the Scorecard and in the 2014 gainful employ- 
ment regulations: a borrower-based rate where the borrower 
must pay down at least $1 of principal balance in the fiscal 
year measured to be counted as in repayment. 49 Repayment 
rates should be calculated at one, three, five, and 10 years into 
repayment in order to maintain consistency with repayment 
rates reported through the College Scorecard, with the length 
of the standard repayment period, and with other post-college 
outcome metrics recommended through the framework. Also, 
for future versions of the framework, field experts and policy- 
makers should define successful repayment as more than just 
a $1 reduction in principal. The field acknowledges the need to 
raise the bar but has not chosen a new threshold for successful 
repayment. 

At a minimum, ED should continue to release repayment rate 
data annually, either through updates to College Scorecard 
data or through the FSA or IPEDS Data Centers, and they also 
should consider improving the quality of the data available in 
the SPRs available to institutions. Detailed recommendations 
regarding repayment rates are discussed in Making Sense of 
Student Loan Outcomes: How Using Repayment Rates Can 
Improve Student Success. 50 
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Use Cases 

To build on current practice, institutions are encouraged to 
integrate the CDR data they receive from ED with student-level 
data in their student information systems in order to conduct 
additional analysis . 51 With this integration, institutions can 
disaggregate default rates by completion, economic status, 
and credential level — including by graduate and undergrad- 
uate student status — to determine which students default. 
With additional support from ED, institutions also can attempt 
to extend the CDR time frame beyond three years, disaggre- 
gate by loan type, and recalculate CDRs based only on debt 
accumulated at their institutions. CDRs are also an important 
consumer information tool for prospective students and fami- 
lies because a high cohort default rate signals that students 
may have a difficult time repaying their loans, and default has 
serious credit consequences for students. Policymakers also 
use CDRs to set basic standards that institutions must meet to 
receive federal financial aid dollars. 


Repayment rates can offer additional actionable data to 
improve borrowers’ post-college outcomes. A set of four insti- 
tutions recently calculated and disaggregated their own repay- 
ment rates in a variety of ways using FSA data and, through 
that endeavor, the colleges noted how valuable the results 
were in helping them rethink their practices, such as student 
loan counseling and financial aid packaging, to set students 
up to repay their debts successfully. When merging FSA data 
with institutional records, institutions can use repayment rates 
to evaluate which borrowers (e.g., noncompleters, students in 
specific programs, low-income students) are least likely to 
make adequate progress on their loans and target financial aid 
and interventions accordingly. Policymakers have proposed 
using repayment rates to enhance institutional standards and 
protect students through either minimum performance thresh- 
olds, risk-sharing, or consumer disclosures. This metric can 
also help alleviate concerns around CDR manipulation and the 
ability of institutions to game the system . 52 
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Graduate Education Rate 

Definition 

The number and percentage of bachelor’s recipients enrolling 
in postbaccalaureate or graduate programs in one, five, and 10 
(optional) years of completion 

Population 

Bachelor’s recipients in a given year 

Disaggregates 

Program of study (at exit), enrollment status, attendance 
intensity (at any time while enrolled), academic preparation (at 
any time while enrolled), economic status (Pell ever), race/ 
ethnicity, gender, age, first-generation status 

Submetrics 
for further 
analysis 

• Relationship between undergraduate program of study and 
graduate program of study 

•Income, gender, or racial gaps in graduate education, 
especially STEM programs 

• Relationship between debt and graduate education 
enrollment or graduate program of study 


Field Usage and Convergence 
In most of the reviewed voluntary initiatives, the graduate 
education rate is not explicitly captured. For some, like the 
NSC and the Western Interstate Commission for Higher 
Education (WICHE), the postbaccalaureate enrollment data 
are collected so the rate can be calculated if the initiatives 
chose to do so. The cohort for this framework’s proposed 
graduate education rate includes only bachelor’s recipients — 
the persistence and transfer metrics capture continuing educa- 
tion rates at 200 percent of program time for certificate- and 
associate’s-seeking graduates. 


Use Cases 

At graduation, bachelor’s recipients have a variety of options. 
In order to comprehensively account for post-college 
outcomes, the framework includes graduate education rates 
to capture outcomes for students who may choose not to 
enter the workforce. For students, policymakers, and institu- 
tions, these rates are used to fill in the gaps that exist when 
only employment outcomes are considered. 

Additionally, institutions can compare graduate education 
rates with the mission of their programs to see if their creden- 
tials are in fact preparing students for their intended 

outcomes — either 
employment or further 
education. If graduate 
education rates are 
not consistent with expected student outcomes (i.e. , the 
continuing education rate is low for a program that should be 
the foundation for graduate school education), then leader- 
ship can evaluate why student pathways are inconsistent with 
the institution’s or program’s goals. Because this metric is 
disaggregated by program of study, institutions also can use 
the submetrics to evaluate the enrollment of specific popula- 
tions of students into graduate programs, specifically for the 
STEM fields, and measure whether students enroll in a 
program similar to that of the undergraduate degree. 


initiatives measure Graduate Education Rate 


The time frames of one, five, and 1 0 years are aligned with the 
earnings and employment metrics to be used in tandem to 
understand the spectrum of post-college outcomes for 
students, furthering the goal of counting all outcomes. To 
further support these timeline thresholds, the Baccalaureate 
and Beyond Longitudinal Study (2008-12) reports that 25 
percent of students surveyed enroll in one year and almost 40 
percent enroll in four years, showing a marked increase 
between the two time frames. 53 Graduate Record Examination 
(GRE) scores are valid for up to five years after taking the test, 
so measuring graduate education after five years should 
capture most students who completed a bachelor’s degree 
with the intention of enrolling in further education. 54 
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Learning Outcomes 

Definition 

Public display of student learning goals, assessments, and 
outcomes using the National Institute for Learning Outcomes 
Assessment’s (NILOA’s) Transparency Framework. 

Institutions also should consider using Lumina Foundation’s 
Degree Qualifications Profile (DQP) and the Association of 
American Colleges & Universities' (AAC&U’s) Valid Assessment 
of Learning in Undergraduate Education (VALUE) Rubrics to 
develop, refine, and measure mastery of learning outcomes 

Population 

Subject to institutional availability 

Disaggregates 

Credential level, economic status (Pell ever), race/ethnicity, and 
academic preparation, program of study 


Submetrics 
for further 
analysis 


• Subject to institutional availability 




initiatives measure Learning Outcomes 


Field Usage and Convergence 

This framework recommends reporting student learning 
outcomes using NILOA’s Transparency Framework as a guide. 
The NILOA Framework encourages institutions to publish 
information about student learning outcomes statements, 
assessment plans, assessment resources, current assess- 
ment activities, evidence of student learning, and use of 
student learning evidence. 55 The Voluntary System of Account- 
ability (VSA), an initiative of APLU and AASCU, recently 
adopted the NILOA Framework. 56 

This framework recommends that institutions consider using 
Lumina Foundation’s DQP and the AAC&U’s VALUE Rubrics 
as tools for developing or refining their approach to student 
learning outcomes. The DQP provides a set of baseline refer- 
ence points for what students should know and be able to do 
on earning a credential at the associate’s, bachelor’s, and 
master’s degree levels. These encompass demonstrating 
proficiency in specific areas of learning including Specialized 
Knowledge, Broad and Integrative Knowledge, Intellectual 
Skills, Applied and Collaborative Learning, and Civic and 
Global Learning. The complementary Tuning process helps 
institutions to identify and assess discipline-specific learning 
outcomes. 57 

VALUE offers a set of rubrics for 16 essential learning 
outcomes. Two of these rubrics — Critical Thinking and Written 
Communication — were endorsed for use as part of the initial 
guidelines for the VSA College Portrait. 58 Lumina Foundation 
also leverages the VALUE rubrics in the DQR highlighting the 
resource as a means for institutions and instructors to under- 
stand student achievement of college-wide learning or course 
objectives. 59 To continue to advance the use of these rubrics 
and to meaningfully compare the results across entities, 
AAC&U is partnering with the State Higher Education Execu- 
tive Officers (SHEEO) to implement them at 69 institutions in 
10 states. 60 


While NILOA, DQR and AAC&U’s VALUE rubrics are continuing 
to advance the field in assessing student learning, institutional 
usage of rubrics and assessments to gauge learning outcomes 
varies widely. Some institutions use these initiatives’ rubrics 
and guidelines; some use standardized tests such as the 
Collegiate Learning Assessment (CLA), ACT’s Collegiate 
Assessment of Academic Proficiency, and the Educational 
Testing Service (ETS) Proficiency Profile; and some use a 
combination of measurement techniques, including those 
developed at the individual institution. 61 As such, clear conver- 
gence across many institutions and initiatives is not yet 

apparent. Because the field 
remains in flux on this topic, 
this framework defers to 
VSA and their use of 
NILOA’s Transparency Framework, as it is used by a large 
array of schools participating in the initiative. 62 

Use Cases 

Learning outcomes strive to quantify what students learn 
through their credential program. States and institutions 
should use these rubrics and assessment tools to benchmark 
progress on student outcomes and to refine teaching and 
curriculum to improve student learning. Institutions can use 
these tools to understand where gaps in student learning exist, 
especially for specific student groups (e.g., low-income 
students and students of color), restructure and revise course 
structure and content, and continuously improve student 
academic achievement. 

Learning outcomes assessments also are used by institutions 
to demonstrate educational effectiveness transparently, effec- 
tively communicate program goals and outcomes to a variety 
of audiences, and fulfill accreditation requirements. While not 
in use in federal data collections, learning outcomes data can 
be used by the institution and state to measure the quality of 
programs and institutions of higher education. For example, in 
2012 and 2013, Massachusetts commissioned the Multi-State 
Collaborative for Leaning Outcomes Assessment to compare 
outcomes with other states in partnership with AAC&U and 
SHEEO. 63 Using the VALUE Rubrics as a common language, 
colleges and universities in Massachusetts used several 
metrics to create composite indicators of student learning, 
including: pass rates on national licensure exams and mean 
scores on graduate entrance exams. 64 States and institutions 
use these exams as evidence that college students accumu- 
lated knowledge and skills while enrolled. Precollege and 
post-college scores are examined to gauge quality of learning 
and inform curricular or instructional changes. 
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Workforce Outcomes: 

Employment Rate, Median Earnings, and Earnings Threshold 

Definition 

Employment rate: The percentage of former students with any 
reported annual earnings at one, five, and 10 years after exit 
from the institution 65 

Median earnings: The median annual earnings of former 
students one, five, and 1 0 years after exit from the institution 
(excludes zeros) 

Earnings threshold: The percentage of former students 
earning more than the median high school graduate salary 
($25,000 in 2014; includes zeros) at one, five, and 10 years 
after exit from the institution. The threshold should be updated 
annually using Current Population Survey data. 66 

Population 

All students who exited the institution in a given year 

Disaggregates 

Credential level, completion status, program of study (at exit), 
economic status (Pell ever), race/ethnicity, gender, age, 
enrollment status, attendance intensity (at any time while 
enrolled), academic preparation (at any time while enrolled), 
first-generation status 


Submetrics 
for further 
analysis 


• Percentiles for earnings (10th, 25th, 75th, and 90th) 
■ Pre- and post-college earnings 

• Relative wages (e.g., compared with local or 


regional wages) 




3 ) initiatives measure Employment Rate 


Field Usage and Convergence 
Increasingly, states and institutions are lever- 
aging workforce data to enhance their under- 
standing of labor markets in relation to higher 
education and student outcomes. Though some 
remain skeptical about the use of workforce 
measures — arguing they show a one-dimensional, incomplete 
view of college outcomes 67 — many seek this return-on-invest- 
ment information. In fact, the use of Unemployment Insurance 
(Ul) wage records at the state level to measure and publish 
student outcomes 68 and the inclusion of earnings and employ- 
ment data in the recently revamped College Scorecard shows 
that students, policymakers, and the public are interested and 
invested in these data. Indeed, at least 90 percent of students 
say a primary reason they are attending college is to get a 
better job, 69 an understandable goal given the substantial 
financial investment students must make in their education. 
Proposed improvements to existing workforce data, like the 
inclusion of major-level data, could help to ease some of these 
concerns. 70 

After years of experimentation with workforce data in the 
states, the field is coming closer to consensus on how to 
define workforce metrics. In recent years, the federal govern- 
ment has built on the work of the states and defined and 
reported postsecondary workforce measures through both 
Gainful Employment and the College Scorecard. This frame- 
work heavily leverages the College Scorecard definitions, with 
some adjustments, to propose two workforce performance 
metrics (employment rate and median earnings ) and one 
workforce efficiency metric (earnings threshold). 


~l j initiatives measure Median Earnings 


Employment Rate 

A variety of demographic data sources, such as the Bureau of 
Labor Statistics, capture employment rates (or often, unem- 
ployment rates) at the national, state, and regional levels. The 
Workforce Innovation and Opportunity Act of 2014 (WIOA) 
requires reporting of employment rates for program exiters in 
addition to median earnings, credential attainment, measure- 
able skill gains, and employer engagement. 71 However, until 
recently employment outcomes have not been reported for 
most colleges, including by program of study. 72 Now, the 
Gainful Employment regulations require institutions to report 
the program-level job placement rates of their graduates, and 
the College Scorecard data include an institution-level unem- 
ployment rate. 73 

Building on these efforts, this framework recommends an 
employment rate that counts as employed any graduate with 
any annual earnings in the specified period. This methodology 

is similar to that used by the 
College Scorecard 74 and the 
metrics reported for Florida 
as part of College Measures’ 
Economic Metrics. 75 Addi- 
tionally, some outcomes- 
based funding models use 
an unemployment or 
employment rate to illustrate 
similar points — the rate at which former students do or do not 
gain employment after exiting the institution or program. 76 

Median Earnings 

This framework also recommends adopting a median earn- 
ings metric to gauge how students fare in the workforce after 
leaving college. A number of initiatives — including College 
Measures, the State Council for Higher Education in Virginia, 
WICHE’s Multistate Longitudinal Data Exchange (MLDE), and 
the College Scorecard — measure post-college earnings, 
reflecting growing interest in these results. 77 This framework 
recommends using median rather than mean earnings to 
follow field practice, 78 disaggregating by at least credential 
level and completion status, reporting on wages at one, five, 
and 10 years after exiting the institution, and adjusting for infla- 
tion. 79 The proposed submetrics can provide added context to 
median earnings information, as follows: 

• Evaluating earnings percentiles as submetrics can provide 
added insight about the full income distribution beyond 
what the median can show. 

• Measuring the change in earnings pre- and post-college can 
provide a better understanding of the value-add of the 
credential, especially for returning adult students. A variety of 


1 ) initiative measures Earnings Threshold 
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initiatives and efforts, including the VFA, the Voluntary Institu- 
tional Metrics (VIM) Project, and Salary Surfer for California’s 
Community Colleges, measure pre- and post-college earn- 
ings. 80 These data are valuable, but institutions are reliant on 
readily available labor data and may be limited in their ability 
to calculate the metric. As data systems improve, the frame- 
work recommends revisiting the possibility of calculating 
pre- and post-earnings metrics on a larger scale. 

• Contextualizing median earnings with regional-level 
economic data can ease concerns about earnings that vary 
based on region. An example of such contextual ization is 
the Aspen Institute College Excellence Program’s measure- 
ment of relative wages, which compares graduates’ wages 
with the average wage of the county of the community 
college, one and five years after program completion. 81 
Earnings Threshold 

Finally, the framework recommends an earnings threshold 
metric based on the methodology of the 2015 College Score- 
card. This threshold measures the proportion of former 
students who earn above a bare minimum amount. ED chose 
$25,000 because it is the median salary of students with a high 
school diploma or GED, 82 so earnings above $25,000 indicate 
a fiscal value-add from the postsecondary education. The 
framework recommends updating this threshold annually to 
account for inflation and national income variation. It works 
particularly well alongside the median earnings metric because 
it sets a baseline expectation that any postsecondary 
program — regardless of expected career path — should at 
least boost student earnings above what they would likely 
earn without the college credential. 83 

Data Sources 

Unfortunately, institutions cannot implement all of these 
recommendations on their own, since they must rely on the 
data provided to them by state and federal sources, which 
need to be refreshed on a regular basis. Routine use of 
employment rates, median earnings, and the earnings 
threshold is contingent on the federal government continuing 
to supply these data through releases like the College Score- 
card. Without them, institutions will need to rely on state 
sources, making it more difficult to achieve measurement 
consistency and comprehensiveness. For instance, while Ul 
wage records are widely used for workforce outcomes, they 
exclude the self-employed, those who work for the federal 
government or military, and former students who reside in a 
different state. 84 Initiatives like WRIS2, the WICHE MLDE, 85 and 
the Federal Employment Data Exchange System build link- 
ages between systems to fill some of these gaps, but a single 
federal source would be a simpler solution. 


However, College Scorecard data are not without limitations. 
These metrics are populated by linking education records 
from NSLDS with earnings data from the U.S. Department of 
Treasury. 86 As a result, the metrics are limited to Title IV aid 
recipients, which represent 70 percent of all college students 
and only 62 percent of community college students. 87 Unfortu- 
nately, without more comprehensive student-level data, the 
federal government is limited to calculating workforce 
outcomes only for students included in NSLDS. 

This framework proposes using these metrics with currently 
available data, but continuing to work toward better metrics, 
which would expand on the recent College Scorecard efforts 
by including all students; disaggregating by credential level, 
program, and completion status; and reporting outcomes one, 
five, and 10 years after program exit — rather than six and 10 
years after program entry. Earnings can vary widely depending 
on college major, as some professions (e.g., teaching, social 
work) are expected to have lower earnings than others (e.g., 
finance, engineering), making program-level disaggregates 
particularly important. 88 Similarly, students who complete 
credentials typically earn more than those who do not, so 
results could be muddied if completers and noncompleters 
are not reported separately. Finally, measuring workforce 
outcomes among a cohort of students who exit at the same 
time would provide more consistent data than for a cohort of 
students who enter at the same time, likely leaving the institu- 
tion at different times due to stopouts and varying time-to- 
degree. The framework recommends following states’ leads 
by measuring workforce outcomes one, five, and 10 years 
after exit, as opposed to six and 1 0 years after entry. 89 

Use Cases 

Post-college workforce outcome measures like earnings, 
employment, and earnings thresholds can be used by a variety 
of audiences. Students and families can use these data to 
learn about the potential earning power of their intended 
degree post-graduation, considering the expected value in 
relation to the major investment required to attend an institu- 
tion of higher education. In recent years, policymakers at both 
the state and federal levels have used workforce outcomes 
data for accountability and funding. For example, gainful 
employment incorporates student earnings — as it relates to 
debt — into its accountability framework. Of the 30 states with 
outcomes-based funding models, 12 use a form of labor 
market outcomes metrics as part of the equation, 90 highlighting 
the importance of these metrics to both policymakers and 
institutions. Institutions can use these data to be aware of their 
students’ outcomes to revise program offerings, tailor prices 
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and financial aid, and implement student supports like career 
services and increased work opportunities that make their 
students more prepared for the workforce. 

The primary reason many students pursue college is to improve 
their employment prospects . 91 While students also gain other 
life skills in college that allow them to contribute to society in 
other nonfinancial ways, a baseline assumption for many 
students it that they will be prepared to earn a middle-class 
living. These metrics can be used individually or in tandem to 
explore post-college workforce outcomes for students. 
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CHAPTER 4: 



ciency Metrics 



This chapter details the following set of efficiency metrics : 1 


Access 4.4 

Expenditures per Student 4.4 

Progression 4.5 


Cost for Credits Not Completed 4.5 

Cost for Completing Gateway Courses 4.6 

Change in Revenue from Change in Retention 4.7 


Completion 4.8 

Cost of Excess Credits to Credential 4.8 
Completions per Student 4.9 

Cost 4.10 

Student Share of Cost 4.10 
Expenditures per Completion 4.11 
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Efficiency Metrics 

While much of this framework focuses explicitly on student 
access to, progression through, and completion of higher 
education, resource-related metrics help provide additional 
context to inform efforts to effectively and efficiently improve 
college attainment. Increased attainment and parity in educa- 
tional outcomes are the ultimate goals of an improved postsec- 
ondary system; moving students through the system more 
efficiently is a supporting goal, as many institutions must better 
serve students with the limited resources already at their 
disposal. Thus, efficiency metrics can help institutions evaluate 
how effectively they are using existing funds to educate students. 

The efficiency metrics included in this framework are based 
mostly (but not exclusively) on the metrics and methodology 
developed by the Delta Cost Project, which uses publicly avail- 
able data from the Integrated Postsecondary Education Data 
System (IPEDS). 2 Because Delta Cost is at the forefront in 
analyzing college and university finance data, many efforts 
have drawn on their work. For example, the National Gover- 
nors Association (NGA) developed its set of efficiency metrics 
for state policymakers based on Delta’s data. 3 Additionally, as 
states embrace outcomes-based funding, many have incorpo- 
rated similar efficiency measures into their funding structures. 4 
Of the initiatives reviewed for this framework, Complete 
College America (CCA) and Completion by Design (CBD) also 
include efficiency metrics in their data reporting. 


The drawbacks to IPEDS finance data are well-known. 5 For 
example, the expenditures data in IPEDS combine undergrad- 
uate, graduate, and noncredit expenditures, inflating any esti- 
mates that attempt to focus on credential-seeking 
undergraduates alone, as this framework does. Further, 
parent/child issues are known to affect institutions that report 
finance data for more than one campus. 6 However, the field 
has not yet developed efficiency metrics, nor a publicly acces- 
sible database, that improve on current practice, so IPEDS 
and Delta Cost data remain the most accurate finance infor- 
mation currently available. The framework also recommends 
adjusting dollars for inflation for metrics that trend over time. 

Institutions can extract much of the data for these metrics 
directly from the Delta Cost Project, which relies on IPEDS data. 
Table 4-1 notes the variables and label names from Delta Cost 
that correspond to the calculations in this chapter. If data 
elements are not available through Delta Cost, or this frame- 
work’s recommendation differs from Delta Cost, those details 
are noted throughout the chapter. For example, neither Delta 
Cost nor IPEDS includes data on credits attempted and 
completed, so institutions must gather it from their student infor- 
mation system. In addition, Delta Cost and IPEDS data on reten- 
tion are based on fall enrollment, not the 12-month enrollment 
cohorts recommended by this framework. 
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Table 4-1: Framework Metrics Mapped With Delta Cost Variables 


Framework’s 
Proposed Metric 

Data Elements Needed to 

Calculate Proposed Metric 

Data Source and Corresponding Variable 

Expenditures per 
Student 

Expenditures for instruction and student 
services 

Expenditures for instruction: current year total (instructionOI) 

Expenditures for student services: current year total (studservOI ) 


Expenditures for instruction, student services, 
research, and public service 

Expenditures for instruction: current year total (instructionOI) 

Expenditures for student services: current year total (studservOI) 

Expenditures for research: current year total (researchOI) 

Expenditures for public service: current year total (pubservOI) 


Expenditures for academic support, institutional 
support, and operations and maintenance 

Expenditures for academic support: current year total (acadsuppOl) 

Expenditures for institutional support: current year total (instsuppOl ) 

Expenditures for operations and maintenance of plant: current year total (opermainOI) 


12-month full-time equivalent (FTE) enrollment 

Total 12-month FTE student enrollment (fte12mn) 


Education and related expenditures 

Education and related expenses (eandr) 

Cost for Credits Not 
Completed* 

Number of credits not completed by cohort in 
the first year 



Number of students attempting credit-bearing 
courses in the cohort in the first year 


Cost for Completing 
Gateway Courses* 

Number of development and gateway course 
credits attempted before completion 



Number of gateway completers in a given year 


Change in Revenue 

Net tuition revenue 

Net tuition and fees revenue (nettuitionOI) 

From Change in 
Retention 

Change in retention rate 

Full-time retention rate (ftretention_rate) for each cohort year** 

Part-time retention rate (ptretention_rate) for each cohort year 


Number of student in cohort 

Full-time first-time degree/certificate-seeking undergraduate for current year (grscohort)** 

Part-time first-time degree/certificate-seeking undergraduate for current year (pt_ugentering) 

Cost of Excess 
Credits to 

Credential* 

Total earned credits for each completer with 
excess credits 


Average credits across all completers 



Total number of completers with excess credits 


Completions per 

Completions per 100 FTE students 

Total completions (totalcompletions) 

Student 


Total 12-month FTE student enrollment (fte12mn) 

Student Share of 
Cost* 

Net tuition revenue paid by students 

Net tuition directly from students (net_student_tuition) 

Expenditures per 
Completion* 

Expenditures per completion 

Education and related expenses per completion (eandr_completion) 


Note: Blank rows in the Data Source column indicate that the variable is to be determined by the institution. 

* Uses education and related expenditures (per credit) as part of calculation, itemized in Expenditures per Student variable. 

** Delta Cost and IPEDS data do not break out first-time and transfer students and are available for only fall entrant cohorts, not 12-month enrollments. 
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ACCESS 


Expenditures per Student 

Definition 

Education and related (E&R) expenditures per full-time 
equivalent (FTE) student based on 12-month enrollment 

Population 

Twelve-month FTE enrollment calculated using 12-month 
instructional activity credit hours in IPEDS 

Disaggregates 

Disaggregations (including separating undergraduate from 
graduate students) are not available due to finance data 
limitations. 


Submetrics 
for further 
analysis 


• Distribution of students by credential level or program of 
study 

• Instructional expenditures per FTE student and as a 
percentage of E&R expenditures 

• Salaries as a percentage of instructional expenditures 

• Student support expenditures per FTE student and as a 
percentage of E&R expenditures 

•Administration expenditures per FTE and as a percentage of 
E&R expenditures 

• E&R expenditures as a percentage of total education and 
general expenditures 


• FTE faculty/staff per FTE student 




The metric also relies on data already available from IPEDS 
that calculate 12-month full-time equivalent (FTE) enrollment 
based on credit-hour activity rather than headcounts 10 
because full- and part-time enrollment is not currently avail- 
able in IPEDS’ 12-month enrollment survey, as this frame- 
work recommends. 

Education and related Expenditures 


Field Usage and Convergence 

This metric relies on IPEDS data and Delta Cost’s methodology 
for calculating E&R, which was developed as a measure of 
spending on direct educational costs. Per Delta Cost, E&R 
includes spending on instruction, spending on student services, 
and a prorated share of spending on academic and administra- 
tive support and operations and maintenance (e.g., administra- 
tion). 7 To determine the prorated amount of indirect spending, 
Delta first finds the “education share” as the spending for 
instruction and student services divided by the spending for 
instruction, student services, research, and public service. 


12-month FTE Enrollment 

However, Delta Cost does not employ 12-month enrollment in 
their trend analyses, which date back to 1998, relying instead 
on fall FTE enrollment because 12-month enrollment was not 
available in IPEDS before 2004. The 12-month enrollment data 
are recommended here to be more inclusive of all students, and 
because more than 1 0 years of trend data are probably suffi- 
cient for most institu- 
tions today. Trend 
analyses of this metric, 
or any cost or effi- 
ciency metric, should be inflation-adjusted overtime. 


initiatives measure Expenditures per Student 


Education Share = 


Expenditures for Instruction and Student Services 

Expenditures for Instruction, Students Services, 
Research, and Public Service 


The education share then is multiplied by the spending for 
academic support, institutional support, and operations and 
maintenance, which is then added to spending on instruction 
and student services, resulting in the E&R. 


In other words, Delta defines E&R as all spending on instruc- 
tion and student services, plus a prorated share of indirect 
expenditures that support the academic mission (e.g., 
academic support, institutional support, and operations and 
maintenance). 8 Because of changes in IPEDS survey reporting 
formats, Delta has made adjustments to reported data in some 
years to maintain comparability overtime. 9 

Expenditures for \ 
Academic support, 


Use Cases 

This metric provides a basic understanding of how much insti- 
tutions are spending to provide an education for each student, 
which is useful to both institutions and policymakers. Colleges 
also can use these data to track trends in their spending per 
student over time and in relation to peer institutions. The 
recommended submetrics (also available from Delta Cost) 
can help colleges determine how changes in spending over 
time impact resource allocation to core educational functions, 
such as instruction and student services, which can help 
contextualize changes in student completion rates. When 
interpreting trends in expenditures per student, institutions 
should evaluate whether changes in the metric resulted from 
changes in enrollment, changes in expenditures (or available 
revenues), or both, for better interpretation and use. For 
students, this metric is not usually a concern or consideration 
in the decision-making process, but may be indicative of how 
much an institution makes available to spend on students rela- 
tive to other institutions. It also can be useful for policymakers 
in clarifying the causes of price increases. It is a widely held 
belief that increases in student tuition and fees are the result of 
surges in college spending, but analysis from the Delta Cost 
Project shows that institutional spending has not risen as fast 
as prices. Rather, they find that a decrease in public subsidies 
is a primary contributor to price increases. 11 


Education 
and related = 
expenditures 


Expenditures 
for Instruction 
and Student 
Services 


Education 

Share 


Institutional support, 
and Operations and 
Maintenance 
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Cost for Credits Not Completed 

Definition 

The per-student expenditures by the institution for credits 
attempted but not completed by first-year students 

Population 

Twelve-month cohorts of first-year students (e.g., first-time 
full-time [FTFT], transfer full-time [TFT], first-time part-time 
[FTPT], transfer part-time [TPT]) 

Disaggregates 

Credential level, academic preparation, economic status, 
race/ethnicity, gender, age, first-generation status, and 
program of study 


Submetrics 
for further 
analysis 


• Number of credits attempted, not completed 

• E&R per credit 

•Total E&R for credits attempted, not completed 

• Total and average net tuition paid by students for 
uncompleted credits 




Field Usage and Convergence 

The cost for credits not completed metric is built on the credit 
completion ratio performance metric and illustrates the mone- 
tary impact for the institution of credits attempted but not 
completed. Prior credits attempted from Advanced Placement 
(AP), International Baccalaureate (IB), dual enrollment, and 
transfer are not counted; however, credits attempted through 
remedial coursework are counted, following the definition for 
the Credit Completion Ratio metric. 


Use Cases 

Institutions waste money and students lose money when 
students attempt credits but do not complete them — whether 
they withdraw from the course or receive failing grades. 13 
Institutions may be able to save or at least better allocate 
scarce resources by improving student course completion. 14 
To interpret accurately the metric’s results, institutions should 
also consider the recommended submetrics, which indicate 
whether the metric is changing over time because of changes 
in the number of uncompleted credits, the expenditures per 
credit, the number of students attempting credits, or all of 

these factors. 
This metric may 
also be of 
interest to state 

and federal policymakers concerned with whether public 
funds are being used to subsidize multiple course repeats. 


initiatives measure Cost for Credits Not Completed 


The framework recommends building the metric calculation 
based in part on Delta Cost Project metrics and methodology, 
using IPEDS finance data. Dividing the E&R expenditures 
(discussed in more detail in the Expenditures per Student 
metric) by all 12-month credit-hour activity provides the E&R 
spending per credit. 12 


E&R per credit is then multiplied by the number of credits not 
completed in the first year, which is divided by the number of 
students attempting credit-bearing courses in the first year. 

Education and related expenditures per credit 
x Number of credits not completed by cohort in the first year 

Number of students attempting credit-bearing courses 
in the cohort in the first year 
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Cost for Completing Gateway Courses 

Definition 

For all gateway course completers in a given year, the 
per-student expenditures associated with all developmental 
and gateway courses attempted before gateway course 
completion, tracking English and math courses separately 

Population 

Gateway course completers in a given year 

Disaggregates 

Credential level, academic preparation (at any time), enrollment 
status, attendance intensity (at any time), economic status (at 
any time), race/ethnicity, gender, age, first-generation status, 
and program of study 


Submetrics 
for further 
analysis 


• Net tuition cost to students to complete gateway courses 

• Enrollment in developmental courses (if applicable) 

• Completion of developmental courses (if applicable) 

• Number of developmental course attempts (if applicable) 

• Enrollment in gateway courses 

• Number of attempts to complete gateway courses 

• Completion of both gateway courses 

•Availability of developmental and gateway courses in 
sequence 

• Percentage of D’s, F’s, W’s, I’s in gateway courses 

• E&R expenditures per credit ^ 



However, this efficiency metric, unlike the performance metric, 
is not cohort-based. Instead, it looks retrospectively at gateway 
completers to examine the efficiency of students’ pathways to 
earning gateway course credit. The efficiency of these path- 
ways is related to, but distinct from, the percentage of students 
completing gateway courses in a given cohort. Interpretation 
of the metric should evaluate whether efficiency is changing 
over time because of changes in the number of gateway 
credits attempted per student, the number of students 
completing gateway courses, the expenditures per credit, or a 
combination of factors. The disaggregates and submetrics 
align with the framework’s 12-month cohort specifications and 
encourage a deeper dive into the institutional costs to help 
students complete gateway courses. 

Use Cases 

Frequent failed attempts at prerequisite remediation or gateway 
courses require institutions to spend money delivering courses 


1 ] initiative measures Cost for Completing Gateway Courses 


Field Usage and Convergence 
Drawing on IPEDS data and Delta Cost methodology, this effi- 
ciency metric largely follows the CBD definition to evaluate the 
cost associated with students passing math and English 
gateway courses — including the expense associated with 
course repeats. 15 Specifically, this metric measures the cost of 
all coursework undertaken to lead to gateway completion — 
including developmental courses in that subject, as well as all 
gateway attempts that eventually led to course completion. 
This metric provides a financial lens that complements the 
metric that measures the percentage of students completing 
gateway courses in the first year. 


that do not result in credit accumulation, and this decreases insti- 
tutional efficiency. These failed attempts also require students to 
spend money and financial aid that does not help them progress 
toward a degree. Quantifying the cost of the various steps toward 
completing a gateway course can help institutions focus on ways 
to increase efficiency and decrease both student and institutional 
expenses. Federal and state policymakers, who subsidize devel- 
opmental and gateway course attempts, also have a vested 
interest in students progressing toward completion in an efficient 
manner, especially in light of time limits on Pell Grant and Direct 
Loan eligibility. 


Number of developmental and gateway 
course credits attempted prior to completion 
x Education and related expenditures per credit in the year attempted 16 

Number of gateway completers in a given year 
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PROGRESSION, continued 


Change in Revenue From Change in Retention 

Definition 

The impact of changes in first-year retention rates from one 
cohort to another on tuition revenue available to the institution 

Population 

Twelve-month cohorts of students (e.g., FTFT, FTPT, TFT, TPT) 

Disaggregates 

Credential level, enrollment status, attendance intensity, 
academic preparation, economic status, race/ethnicity, gender, 
age, first-generation status, and program of study 


Submetrics 
for further 
analysis 


• Change in first-year retention rates over time 

• Change in net tuition per student over time 

• Change in net tuition revenue per student due to change in 
retention 

• Change in subsidy revenue due to change in retention (total 
and per student) 

• Change in net tuition plus subsidy revenue due 
to change in retention (total and per student) 


Use Cases 

By highlighting the possibility for increased revenue genera- 
tion resulting from retention rate increases, the results of the 
metric can support institutions advocating for additional 
funding for student support services that improve retention. 
The metric also can quantify the return on investment of those 
support services, which can ultimately offset some of their 
costs . 21 Considering the investment of the state and federal 
governments in higher education, these data can help policy- 
makers to understand and support efforts to increase student 
retention because of the impact these efforts have on institu- 
tional, state, and federal budgets. 


initiatives measure Change in Revenue From Change in Retention 


Field Usage and Convergence 

The change in revenue from change in retention metric is 
related to the retention rate performance metric in that it 
compares first-year retention rates for two cohorts and evalu- 
ates how that change impacts net tuition revenue for an institu- 
tion. CBD includes a similar efficiency metric as an optional 
measure for use by its community colleges, but variations on 
this measure have been used widely by the field . 17 


Change in the 
first year retention 
rate from Cohort 1 
to Cohort 2 


Number of 
students in 
Cohort 2 


Net tuition 
revenue per 
student for 
year 2 la 


Similar to CBD, this metric calculates changes in net tuition 
revenue only, related to changes in retention, because institu- 
tions receive tuition on a per-student basis, whereas other 
funding sources may be allocated in a variety of ways . 19 
However, based on recent studies, institutions might also 
consider estimating the impact of changes in retention on 
other revenue sources, such as local and state appropria- 
tions . 20 To fully understand the results, institutions must deter- 
mine whether the revenues are changing over time because of 
changes to the number and percentage of students retained, 
changes in net tuition, or both. When trending these data over 
time, it is essential to adjust for inflation and consider changes 
in the size of cohort as well as in tuition for accurate analysis. 
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COMPLETION 


Cost of Excess Credits to Credential 

Definition 

The per-student expenditures for excess credits to credential 
for all completers with excess credits in a given year 

Population 

All completers in a given year by credential level 

Disaggregates 

Enrollment status, attendance intensity (at any time), academic 
preparation (at any time), race/ethnicity, economic status (at 
any time), age, gender, program of study (at exit) 


Submetrics 
for further 
analysis 


• Excess credits earned by transfers by number/percentage of 
prior credits accepted 

•Total (instead of average) cost of excess credits to credential 

• Total and average net tuition cost to student of 


excess credits to credential 




Use Cases 

This metric measures the financial outlay by the institution for 
students taking excess credit hours to credential. Because of 
the multitude of factors affecting this metric, it is imperative to 
determine whether efficiency is changing due to more students 
taking more excess credits, the expenditures per credit, or 
both over time. Changes in expenditures per credit over time 
can be controlled for by using the expenditures per credit in 
the year the credit was taken, instead of the year the student 


initiatives measure Cost of Excess Credits to Credential 


Field Usage and Convergence 

The metric is related to the average credits to credential metric 
in this framework, which mostly follows CCA calculation guide- 
lines to determine the average number of credit hours 
completed by credential earners by credential level. The 
framework goes further by calculating the number of credits 
earned above the average number of credits, to calculate 
excess credits to credential, based on CBD’s optional average 
number of excess credits metric. This framework’s proposed 
efficiency metric combines these approaches with IPEDS data 
and Delta Cost methodology by assigning costs to the excess 
credits. 


completed, for more precision. If costs increase largely due to 
excess course taking, institutions can proactively address 
degree pathways and academic advising to improve efficiency 
and help students complete more quickly and at a lower cost. 
The metric also provides institutions and policymakers with 
another piece of the cost-of-college puzzle, identifying a 
possible intervention strategy to reduce costs for both students 
and taxpayers. By creating efficient pathways to a credential, 
institutions and students can minimize excess credits to 
credential, lessening the cost per completer for the institution, 
student, and taxpayer. 


It does so by calculating the total number of credits earned 
above the average, summed across completers who accumu- 
lated more than the average. Multiplying by expenditures per 
credit (E&R expenditures divided by 12-month instructional 
credit activity) provides the total cost of these excess credits, 
and dividing by the number of completers with excess credits 
translates this metric into a cost per completer (with excess 
credits). Using the average credits to credential to calculate 
excess credits includes some inefficiency because many 
students may take more credits than required, thus increasing 
the average above catalog credit requirements. Ideally, this 
metric would use the number of credits required by the course 
catalog for each credential level and program of study instead 
of the average number of credits to credential. However, that 
level of detail currently is not readily accessible to institutional 
researchers based on our discussions with the field. As such, 
this metric should be interpreted with caution as some of the 
“excess” may be explained by requirements that differ across 
programs and degrees. 
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COMPLETION, continued 


Completions per Student 

Definition 

The number of completions divided by the number of 

FTE students (based on 12-month enrollment) in a given year 
expressed as completions per 100 FTE 

Population 

Twelve-month FTE undergraduate enrollment and 
undergraduate credentials (certificates, associate's, bachelor’s) 
conferred in a given year 

Disaggregates 

Race/ethnicity, gender, age, credential level, program of study 
(at exit), academic preparation (at any time), economic status 
(at any time), first-generation status, enrollment status 
(at entry), attendance status (at any time) 


Submetrics 
for further 
analysis 


• Change in number of completers 
•Change in FTE enrollment 

• Completion rates 




Field Usage and Convergence 

Simply stated, this metric is intended to show how effectively 
institutions turn credential-seekers into credential-holders. The 
recommended metric follows Delta Cost Project and CCA 
methodology with some proposed modifications. 22 These 
include using 12-month undergraduate enrollment, instead of 
fall, and disaggregating at least by credential level, race, 
gender, age, economic status, and academic preparation. 
IPEDS collects disaggregations for completions by program of 
study, credential level, race/ethnicity, and gender, but does not 
disaggregate 12-month FTE enrollment by any demographic 
characteristics. 23 While CCA does not currently disaggregate 
completions per student by these student characteristics, they 
could disaggregate this metric by most of the characteristics 
recommended here given the level of disaggregation available 
for both their enrollment and credential production metrics. 24 


When examining this metric over time, it should be noted 
whether changes in the metric are due to changes in the 
number of credentials awarded, the number of FTE students, 
or both, for better interpretation and use. 

Use Cases 

Some institutions use this metric to illustrate student progress 
toward graduation. For example, the University of Texas-EI 
Paso uses a similar degree-production ratio that compares the 
total number of FTE undergraduates enrolled four years earlier 
with the total number of baccalaureate degrees awarded that 

year. 26 These data can 
supplement the tradi- 
tional IPEDS gradua- 
tion rates by capturing 
completions regardless of whether the student began with a 
first-time, full-time status, although the more inclusive comple- 
tion rates recommended as part of this framework can alle- 
viate this issue. Policymakers can also use this metric, in 
conjunction with success rates, to determine how many 
credentials institutions award in relation to how many students 
they enroll. Some states, like Tennessee, include a similar 
completion per 100 FTEs metric in their outcomes-based 
funding models. 27 


initiatives measure Completions per Student 


Completers would be the ideal option for the numerator 
because they count unduplicated students, as opposed to 
completions, which count students more than once if they 
earn multiple credentials in the same year. However, data on 
completers were not added to IPEDS until the 2012-13 collec- 
tion, so these data cannot be trended over a longer span of 
time. 25 Once more data are available for completers, the 
framework recommends amending this metric to calculate 
completers per student, to produce an unduplicated measure. 
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COST 


Student Share of Cost 

Definition 

The percentage of E&R covered by net student tuition revenue 
versus institutional subsidies in a fiscal year 

Population 

All students 

Disaggregates 

Disaggregations (including separating undergraduate from 
graduate students) are not available due to finance data 
limitations. 


Submetrics 
for further 
analysis 


• Sticker price and net price 

•Net tuition revenue per 12-month FTE enrollment 


Use Cases 

Net tuition revenue accounted for roughly 50 to 62 percent of 
education-related spending at public four-year institutions in 
2013. The student share of costs at public institutions was 
between 10 and 19 percentage points higher than it was in 
2003, varying by institution type. 28 The tuition share of E&R at 
private institutions was also about 4 to 5 percentage points 
higher in 2013 than it had been a decade prior. 


•E&R per 12-month FTE enrollment 
•Subsidy per 12-month FTE enrollment 


Field Usage and Convergence 

This metric is drawn directly from the Delta Cost Project, which 
refers to it as the net tuition share of E&R. The metric quantifies 
the proportion of education-related expenditures paid for by 
net tuition revenue relative to other institutional resources, 
such as state and local appropriations, investment or endow- 
ment incomes, or other revenues generated by the institu- 
tion — or what Delta Cost calls the “subsidy share.” 

Net tuition revenue (gross tuition revenue minus institutional grant aid) 
Education and related expenditures 

The metric includes the entire student population, rather than 
only undergraduates, because graduate and undergraduate 
expenditures cannot currently be separated in IPEDS/Delta 
Cost Project data. Further, while no disaggregates are recom- 
mended due to data limitations, the submetrics are intended 
to better explain whether and how burden has shifted to 
students, so that institutions and policymakers may seek to 
lessen the financial impact on students, particularly those with 
the greatest need. 


This metric is highly rele- 
vant to policymakers 
because it quantifies the 
impact of decreased state support for higher education — and 
its direct impact on students. As per-student state investment 
has declined, students and families have had to pick up an 
increasing share of college costs, affecting their ability to 
access and succeed in college, especially for low-income 
students with fewer resources to draw on. 29 A report using 
Delta Cost Project data noted that decreased state funding is 
responsible for almost 80 percent of the rise in public educa- 
tion tuition between 2001 and 201 1 . 30 While more recent anal- 
ysis shows a slight increase in per-student state and local 
funding for public colleges and universities (5.4 percent 
between 2013 and 2014), longer-term trends in state disinvest- 
ment in higher education have had a major impact on college 
affordability. 31 State policymakers should work to restore 
appropriations to at least prerecessions levels, and institutions 
should realign institutional aid practices to address the finan- 
cial hardships of low-income students and families, who were 
unduly burdened by cuts. 




initiative measures Student Share of Cost 
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COST, continued 


Expenditures per Completion 

Definition 

E&R divided by the number of completions in a fiscal year 

Population 

All credentials conferred in a given year 

Disaggregates 

Disaggregations (including separating undergraduate from 
graduate students) are not available due to finance data 
limitations. 


Submetrics 
for further 
analysis 


• Distribution of completions by award level and program of 
study 

• Change in number of completions over time 

• Change in E&R over time ^ 


Field Usage and Convergence 

This metric, also drawn directly from the Delta Cost Project, 
calculates E&R per completion, including all credentials 
because undergraduate and graduate students cannot 
currently be disaggregated in the IPEDS expenditure data. 
Additional disaggregates also are not included for this metric 
due to data limitations, but the recommended submetrics can 
help to explain whether the metric is changing due to changes 
in the number of completions, the level of expenditures, or 
both over time. Further, understanding the distributions of 
completions by award level and program of study can help 
interpret why expenditures at a given institution may be higher 
or lower than at other institutions with different credential and 
program mixes. 


Use Cases 

This metric is a proxy for the resources required to educate 
students through to credential completion. It is a proxy 
because the data are not readily available to assign actual 
costs to individual students as they progress (or do not prog- 
ress) toward completion. As such, this metric captures the 
costs associated with both completers but also noncom- 
pleters, by comparing the resources spent to educate all 

students in a given 
year with the 
number of creden- 
tials awarded by 
the institution in that same year. Initiatives like CBD and the 
Voluntary Institutional Metrics Project already use the expendi- 
tures per completion metric to measure the cost associated 
with achieving the ultimate goal of degree completion. 33 


initiatives measure Expenditures per Completion 


Completers would be the ideal option for the denominator 
because they count unduplicated students, as opposed to 
completions, which count students more than once if they 
earn multiple credentials in the same year. However, data on 
completers were not added to IPEDS until the 2012-13 collec- 
tion, so these data cannot be trended over a longer span of 
time. 32 Once more data are available for completers, the frame- 
work recommends amending this metric to calculate 
completers per student, to produce an unduplicated measure. 
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CHAPTER 5 


Equity Metrics 



This chapter details the equity metrics that are crucial to promoting and enhancing 
equity within higher education by disaggregating the performance and efficiency 
metrics by critical student characteristics. The following characteristics are 
considered in more detail, given the greater complexity required to define them: 


Academic preparation 5.2 

Economic status 5.3 


First-generation status 

Program of study .... 

Race/ethnicity 

Gender 

Age 


5.5 


5.6 


5.6 


5.6 
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Key Student Characteristics/Disaggregates 

A core purpose of data collection and use is to shine a light 
on — and to develop strategies to close — gaps in college 
access and success that continue to disadvantage underrep- 
resented students. Nontraditional and underserved student 
populations have largely been left out of or are invisible in 
federal data collections, making it difficult or impossible to 
measure how well these students are served by higher educa- 
tion and to develop strategies to better serve them. As such, 
this framework recommends disaggregating each metric by 
key student characteristics used by a host of voluntary data 
initiatives over the past decade. These equity-focused disag- 
gregates are essential to uncovering and remedying inequities 
in and across our colleges and universities. 

Depending on the metric type, the framework recommends 
determining student characteristics at different points in time: 
at entry, ever during enrollment, or at exit. The time of identifi- 
cation is shown in the snapshot charts of Chapters 3 and 4. In 
general, the framework follows Complete College America 
(CCA) and Access to Success (A2S) precedent by basing 
student characteristics at entry for cohort-based measures, 
like graduation rates, and defining them if the student met the 
criteria af any time for retrospective measures, such as comple- 
tions. For disaggregates, such as major and credential 
received, which are most relevant at the point of college exit, 
the framework recommends defining them at exit. For cost 
metrics, such as net price and unmet need, that are measured 
annually, the framework recommends defining disaggregates 
at that time , to reflect the student’s status that year. Recom- 
mendations for how to define the student disaggregates — 
including academic preparation, economic status, 
first-generation status, program of study, race/ethnicity, 
gender, and age — are explored below. 


initiatives measure Academic Preparation 


Academic Preparation 

This framework recommends that institutions minimally iden- 
tify students as “college ready” or “not college ready” in math 
and in English according to their own criteria until further 
research develops more robust measures of academic prepa- 
ration that 
are compa- 
rable across 
colleges. 

Often-used proxies for academic preparation include stan- 
dardized test scores, high school GPA, placement or enroll- 
ment in remedial education, and multiple measures frameworks 
that incorporate several metrics. If college-ready assessments 
like the Partnership for Assessment of Readiness for College 
and Careers (PARCC) or Smarter Balanced gain widespread 
use, this recommendation should be revisited to determine 
whether performance on these exams could serve as an 
adequate measure of college-readiness. Because the field has 
not yet converged on a universally accepted indicator for 
college readiness, the framework defers to institutional prac- 
tices until further research shows consensus. 


Field Usage and Convergence 

To determine the most appropriate metric for academic prepa- 
ration, we reviewed current and emerging research around: 
high school curriculum rigor, high school GPA, college 
entrance exam scores, remedial coursework, and multiple 
measures (See Table 5-1). 

Use Cases 

Measures of academic preparation are crucial for institutions 
to understand whether incoming students are ready for a 
college environment; they highly correlate with students’ 
college outcomes without intervention. 9 Colleges and universi- 
ties can use these data to develop and target services to best 
reach underprepared students and create pathways for their 
college success. In addition, academic preparation data allow 
institutions to measure the efficacy of interventions that aim to 
help students become college-ready after entry. Policymakers 
can use academic preparation at the state level to develop 
coherent and consistent policies to signal clearly to students 
and schools how they should prepare for college in terms of 
high school curriculum and remedial education in college. 10 
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Table 5-1: Advantages and Disadvantages to Alternate Academic Preparation Indicators 



Advantages 

Disadvantages 

High School Curriculum Rigor 

Considered the best predictor of college success based on 
quantitative analysis by the Department of Education 1 

Time and labor intensive to quantify, because measuring high 
school rigor requires transcript analysis 

Measurement would be difficult to operationalize at scale because 
of the labor required to implement 

High School GPA 

Considered one of the best predictors of college entrance, 
persistence, and completion through correlation and regression 
analysis 2 

Captures academic performance (cognitive) and personal 
attributes (noncognitive), such as motivation and perseverance 3 

Requires a GPA threshold to define “college-ready,” and though 
there is a linear relationship between high school GPA and college 
outcomes, there are no clear GPA cutoffs to indicate readiness 4 

College Entrance Exam Scores 

Used by many institutions during the admission process to 
determine college readiness 

Both SAT and ACT scores are predictive of first-year GPA and 
student outcomes, such as retention and completion, in college 5 

Many students at open-access institutions, such as community 
colleges and for-profit schools, do not take these tests 

Remedial Coursework 

Used by many initiatives, states, and institutions to signal college 
readiness 6 

Not all institutions offer remediation, and many are shifting away 
from stand-alone courses toward co-requisite remediation models 
that may be more difficult to track 

Remedial placement policies vary widely across states and 
institutions, as there are no shared standards 

The predictive value of taking remedial courses on completion 
varies substantially across credential types, having little predictive 
value for certificate and associate’s seekers, while predictive for 
bachelor’s seekers 7 

Multiple Measures 

Creates a composite view of the student, using a variety of 
indicators such as standardized test scores, high school GPA, and 
high school course completion and rigor 

Deployed in a number of states and institutions 8 

Much of the research is still ongoing; no clear best practices have 
emerged yet but may allow for more robust recommendations in 
the future 


initiatives measure Economic Status 


Economic Status 

The framework recommends using Pell Grant receipt as the 
primary indicator of low-income status at this time, despite its 
known limitations, which are discussed below. Pell receipt is 

the most frequently 
used measure of 
economic status in 
the field, and each 
alternate indicator faces even more substantial limitations than 
Pell receipt. Table 5-2 explores the advantages and disadvan- 
tages of six potential measures of economic status: Pell Grant 
receipt, Pell Grant eligibility, expected family contribution 
(EFC), income, poverty status, and student’s home location 
(geocode). Income is a promising indicator for economic 
status that should be tested further in the field and explored for 
inclusion in future iterations of the framework. 


Field Usage and Convergence 

Higher education can be an engine of social and economic 
mobility, but low-income students remain underrepresented 
among college-goers and college graduates. To promote 
mobility, equity, and our nation’s economic competitiveness, 
many federal, state, local, and institutional efforts center on 


improving access and success for low-income students. For 
instance, all initiatives reviewed as part of this research — 
Completion by Design, A2S, Achieving the Dream, Voluntary 
Framework of Accountability, the new College Scorecard, and 
more — use Pell Grant receipt as an indicator of low-income 
status. 

While Pell receipt is a frequently used proxy for economic 
status, it is not perfectly accurate. Its primary limitation is that 
it undercounts the proportion of low-income students, espe- 
cially at institutions where many do not apply for federal finan- 
cial aid, due to either lack of information, low costs, or 
citizenship status. Also, it is subject to changes in federal 
financial aid policy, sometimes causing notable shifts that may 
not actually reflect demographic shifts." However, Pell receipt 
remains the primary indicator of economic status used by the 
field, is fairly comprehensive of low-income students, and 
takes into consideration important factors that influence finan- 
cial need, such as family size. In 2011-12, 41 percent of under- 
graduate students were Pell recipients. 12 
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Table 5-2: Advantages and Disadvantages to Economic Status Proxies 



Advantages 

Disadvantages 

Pell Grant Receipt 

Most commonly used proxy for low-income status in the field 

Accounts for factors aside from income that influence financial 
need 

Undercounts low-income students, especially at institutions where 
many students do not file a Free Application for Federal Student 

Aid (FAFSA) 

Vulnerable to policy changes unrelated to shifts in economic 
status 

Pell Grant Eligibility 

In addition to Pell recipients, captures low-income students who 
apply and are eligible for but do not receive Pell because of 
administrative hurdles 

Accounts for factors aside from income that influence financial 
need 

Can be determined only for FAFSA completers 

Requires integration of data for non-Pell FAFSA filers in student 
information systems, which could be burdensome 

Vulnerable to policy changes unrelated to shifts in economic 
status 

Expected Family Contribution 

In addition to Pell recipients, captures low-income students who 
apply and are eligible for but do not receive Pell because of 
administrative hurdles 

Expressed in a dollar amount, which is helpful in making 
comparisons with price 

Accounts for factors aside from income that influence financial 
need 

Allows for an “EFC Unknown” category that prevents classifying 
all students with missing data as non-low-income (with a 
dichotomous Pell proxy, some students classified as non-Pell 
simply have missing data) 

Can be determined only for FAFSA completers 

Requires integration of data for non-Pell FAFSA filers in student 
information systems, which could be burdensome 

Vulnerable to policy changes unrelated to shifts in economic 
status 

Requires defining a threshold to determine what EFC counts as 
low-income, although Pell eligibility guidelines can be used 

Family Income 

In addition to Pell recipients, captures low-income students who 
apply for but do not receive Pell for a variety of reasons 

Expressed in a dollar amount, which is helpful in making 
comparisons to price 

Allows for an “Income Unknown” category that prevents 
classifying all students with missing data as non-low-income (with 
a dichotomous Pell proxy, some students classified as non-Pell 
simply have missing data) 

College Scorecard paved the way by calculating completion rates 
of Title IV recipients by family income 

Can be determined only for FAFSA completers 

Requires integration of data for non-Pell FAFSA filers in student 
information systems, which could be burdensome 

Does not take into consideration important characteristics like 
family size, dependency status, and number of family members in 
college 

Requires defining a threshold to determine what income counts as 
low-income, although some guidance is available from national 
Census data 

Poverty Status 

In addition to Pell recipients, captures low-income students who 
apply for but do not receive Pell for a variety of reasons 

Commonly used in means-tested programs 

Can be determined only for FAFSA completers 

Requires integration of data for non-Pell FAFSA filers in student 
information systems, which could be burdensome 


Accounts for one factor that influences financial need aside from 
income (family size) 

Does not take into account dependency status or the number of 
students in college 



Requires defining a threshold to determine which poverty level 
counts as low-income 

Student’s Home Location 
(Geocode) 

Students’ home residence location can be predictive of student 
outcomes, on average 13 

Captures all students regardless of FAFSA completion or aid 
receipt 

Best used to describe an institutional service area and institutional- 
level outcomes, rather than economic status for individual 
students, which may vary widely within the same geographic 
area 14 


As noted in Table 5-2, some indicators could increase coverage 
beyond only the aided students captured by a Pell receipt 
proxy, by counting the low-income students who file a FAFSA 
(and thus have their data recorded) but do not receive a Pell 
Grant — possibly because of administrative hurdles such as 
verification. Table 5-3 examines by how much each option 
undercounts or improves upon other options. Data for the 
analysis are derived from the National Postsecondary Student 
Aid Survey (NPSAS) in 2012, which imputes EFC for students 
who did not file a FAFSA. Analyzing statistics on Pell receipt, 
EFC (including imputed values), income, and poverty level, 


alongside the percentage of students who filed a FAFSA, we 
can calculate the percentage of students that institutions 
should be able to identify as low-income using that indicator of 
economic status — assuming they can discern EFC, income, 
or poverty level only for FAFSA filers. 

For example, while 58 percent of students likely would be Pell- 
eligible based on their (actual or imputed) EFCs in NPSAS, 
only 41 percent receive Pell Grants. However, if Pell eligibility/ 
EFC were used as a proxy for economic status, it would 
increase the percentage of students known to the institution as 
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Table 5-3: Accuracy of Economic Status Proxies 


Indicator 

Threshold 

A = Percentage of Students 
Identified as Low-Income in 
NPSAS 

B = Percentage of Students 
Identified as Low-Income Who 
Filed a FAFSA 

C = Percentage of Students 
Potentially Known as 

Low-Income by Institutions 
(C = A x B) 

Pell Grant Receipt 

Receipt equals low-income 

41% 

100% 

41% 

Pell Grant Eligibility or EFC 

EFC under $5,273 equals 
low-income 15 

58% 

83% 

48% 

Family Income 

Income in the bottom two 
quintiles nationally equals 
low-income 

60% 

78% 

47% 

Poverty Status 

250% of poverty 16 

59% 

79% 

47% 


Source: IHEP analysis of National Postsecondary Student Aid Study 2012 data. 


low-income by only 7 percentage points over Pell receipt (48 
percent vs. 41 percent), because only 83 percent of students 
with Pell-eligible EFCs actually file a FAFSA, which institutions 
rely on to obtain this information. 

So, as Table 5-3 shows, while using Pell eligibility, EFC, family 
income, or poverty status could count slightly more low- 
income students (6-7 percentage points), the added precision 
does not warrant the added complication of diverging from 
how the field typically measures economic status. Further- 
more, the majority (71 percent) of students with Pell-eligible 
EFCs do in fact receive the grants, making it a sufficiently 
accurate — albeit imperfect — proxy. 17 Because of Pell receipt’s 
widespread use and its coverage relative to the other proxy 
variables, the framework recommends Pell receipt as the best 
metric at this time. 

Use Cases 

Institutions can use economic status to disaggregate other 
metrics and gain a better understanding of how low-income 
students are accessing and succeeding in their colleges or 
universities. Low-income students face different challenges in 
higher education than do middle- and high-income students, 
so it is crucial that institutions have access to disaggregated 
data to identify gaps and to tailor solutions and financial aid 
strategies for the neediest students. Recent research confirms 
that some institutions serve low-income populations more 
effectively than others, so institutions can use these data to 
continuously improve student access and success. 18 In addi- 
tion, state and federal policymakers often express interest in 
understanding how low-income students access, progress 
through, and succeed in higher education. At the federal level 
specifically, policymakers are interested in the outcomes of 
low-income students, and a recent Integrated Postsecondary 
Education Data System (IPEDS) proposal includes Outcome 
Measures for Pell Grant recipients. 19 


First-Generation Status 

The framework recommends defining first-generation students 
as students whose parents' highest education level was some 
college but no degree, or below (e.g., some college, no degree; 
vocational/technical training; high school diploma or equiva- 
lent; did not complete high school). Defined as such, first- 
generation students constitute 52 percent of undergraduates. 20 


3 ) initiatives measure First-Generation Status 


Field Usage and Convergence 

According to Beginning Postsecondary Students (BPS) Longi- 
tudinal Study, degree completion rates increase from 35 
percent for students whose parents have no education beyond 
high school, to 56 percent for students whose parents have 
bachelor’s degrees or higher. While there is a linear increase in 
students’ completion rates as their parents’ education level 
increases from high school to some college, to associate’s 
degree, to bachelor’s degree, to professional degree, there is 
a sizable difference between students whose parents have 
less than an associate’s degree (43 percent) and those whose 
parents have an associate’s degree or higher (59 percent). 21 

While the federal TRIO programs, which provide supports to 
low-income students, first-generation students, and student 
with disabilities, identify students as first-generation if their 
parent(s) do not have bachelor’s degrees, 22 current policy 
conversations that focus on baccalaureate and sub-baccalau- 
reate credentials suggest that there is value in shifting the defi- 
nition. Furthermore, the gap in overall degree completion 
between non-first-generation and first-generation students 
increases by only one percentage point when students whose 
parents have associate’s degrees are included in the first- 
generation group. 23 
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The share of first-generation students is also available in the 
College Scorecard as a disaggregate for the student body and 
other measures such as median cumulative debt and earn- 
ings. These data on first-generation status are based on self- 
reported information on the FAFSA. When measuring the 
share of the student body that is first-generation, data are 
reported separately for students whose parents’ highest 
education level is middle school, high school, and some post- 
secondary education. 24 

Use Cases 

Parental education is highly correlated with student outcomes, 
and considerable efforts in the field are focused on improving 
outcomes for this population. Measuring these gaps at the 
institution level can help colleges address them. Many institu- 
tions, states, community-based organizations, and the federal 
government implement programs and student supports 
geared toward first-generation students to assist them in over- 
coming obstacles related to access and completion of a 
college degree. Initiatives like I’m First serve as a resource for 
first-generation students, providing information and peer 
support. 25 Movements by first-generation students on college 
campuses, backed and supported by these institutions, also 
help to create a system of support, especially at institutions 
where the class divide is more apparent. 26 Institutions and poli- 
cymakers need disaggregated data to continue to support 
first-generation students through interventions like the TRIO 
programs and to create an environment where these students 
can succeed. 


Additional Disaggregates 

The remaining disaggregates follow the conventions of most 
reviewed initiatives. 


initiatives measure Program of Study 


Program of study 

Researchers, advocates, and institutions advocate for disag- 
gregation of data by program of study to provide the most 

refined view of 
student outcomes 
possible. Given 
the value of 
program-level data, the framework recommends using the 
Classification of Instructional Program (CIP) codes. Institu- 
tions should collect data at the six-digit CIP code level and 
aggregate to two-digit codes for reporting purposes aligned to 
CCA seven meta-majors: Education; Arts and Humanities; 
Social and Behavioral Sciences and Human Services; Science, 
Technology, Engineering, and Math (STEM); Business and 
Communication; Health; and Trades. 27 


Race/ethnicity 

The framework recommends using the latest IPEDS race/ 
ethnicity categories: Hispanic or Latino; American Indian or 

Alaska Native; Asian; 
Black or African- 
American; Native 
Hawaiian or Other 
Pacific Islander; White, Two or more races; Nonresident alien; 
and Race/ ethnicity unknown. 


initiatives measure Race/ethnicity 


Gender 

The framework recommends using IPEDS gender definitions 

(Male and Female) and 
adding an Other category. 


initiatives measure Gender 


Age 

The framework recommends using date of birth if such data 
are available. Otherwise, we recommend disaggregating by 

age categories aligned with 
CCA: 19 and under, 20-24, or 
25 and over. 


initiatives measure Age 
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Appendix 1: Major Data Initiatives & Measures Crosswalk 
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*These are the disaggregates available for each initiative. Not all of the measures are disaggregated by all characteristics listed here. 
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